{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logistic 回归模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上一节课我们学习了简单的线性回归模型，这一次课中，我们会学习第二个模型，Logistic 回归模型。\n",
    "\n",
    "Logistic 回归是一种广义的回归模型，其与多元线性回归有着很多相似之处，模型的形式基本相同，虽然也被称为回归，但是其更多的情况使用在分类问题上，同时又以二分类更为常用。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 模型形式\n",
    "Logistic 回归的模型形式和线性回归一样，都是 y = wx + b，其中 x 可以是一个多维的特征，唯一不同的地方在于 Logistic 回归会对 y 作用一个 logistic 函数，将其变为一种概率的结果。 Logistic 函数作为 Logistic 回归的核心，我们下面讲一讲 Logistic 函数，也被称为 Sigmoid 函数。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sigmoid 函数\n",
    "Sigmoid 函数非常简单，其公式如下\n",
    "\n",
    "$$\n",
    "f(x) = \\frac{1}{1 + e^{-x}}\n",
    "$$\n",
    "\n",
    "Sigmoid 函数的图像如下\n",
    "\n",
    "![](https://ws2.sinaimg.cn/large/006tKfTcly1fmd3dde091g30du060mx0.gif)\n",
    "\n",
    "可以看到 Sigmoid 函数的范围是在 0 ~ 1 之间，所以任何一个值经过了 Sigmoid 函数的作用，都会变成 0 ~ 1 之间的一个值，这个值可以形象地理解为一个概率，比如对于二分类问题，这个值越小就表示属于第一类，这个值越大就表示属于第二类。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "另外一个 Logistic 回归的前提是确保你的数据具有非常良好的线性可分性，也就是说，你的数据集能够在一定的维度上被分为两个部分，比如\n",
    "\n",
    "![](https://ws1.sinaimg.cn/large/006tKfTcly1fmd3gwdueoj30aw0aewex.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到，上面红色的点和蓝色的点能够几乎被一个绿色的平面分割开来"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 回归问题 vs 分类问题\n",
    "Logistic 回归处理的是一个分类问题，而上一个模型是回归模型，那么回归问题和分类问题的区别在哪里呢？\n",
    "\n",
    "从上面的图可以看出，分类问题希望把数据集分到某一类，比如一个 3 分类问题，那么对于任何一个数据点，我们都希望找到其到底属于哪一类，最终的结果只有三种情况，{0, 1, 2}，所以这是一个离散的问题。\n",
    "\n",
    "而回归问题是一个连续的问题，比如曲线的拟合，我们可以拟合任意的函数结果，这个结果是一个连续的值。\n",
    "\n",
    "分类问题和回归问题是机器学习和深度学习的第一步，拿到任何一个问题，我们都需要先确定其到底是分类还是回归，然后再进行算法设计"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 损失函数\n",
    "前一节对于回归问题，我们有一个 loss 去衡量误差，那么对于分类问题，我们如何去衡量这个误差，并设计 loss 函数呢？\n",
    "\n",
    "Logistic 回归使用了 Sigmoid 函数将结果变到 0 ~ 1 之间，对于任意输入一个数据，经过 Sigmoid 之后的结果我们记为 $\\hat{y}$，表示这个数据点属于第二类的概率，那么其属于第一类的概率就是 $1-\\hat{y}$。如果这个数据点属于第二类，我们希望 $\\hat{y}$ 越大越好，也就是越靠近 1 越好，如果这个数据属于第一类，那么我们希望 $1-\\hat{y}$ 越大越好，也就是 $\\hat{y}$ 越小越好，越靠近 0 越好，所以我们可以这样设计我们的 loss 函数\n",
    "\n",
    "$$\n",
    "loss = -(y * log(\\hat{y}) + (1 - y) * log(1 - \\hat{y}))\n",
    "$$\n",
    "\n",
    "其中 y 表示真实的 label，只能取 {0, 1} 这两个值，因为 $\\hat{y}$ 表示经过 Logistic 回归预测之后的结果，是一个 0 ~ 1 之间的小数。如果 y 是 0，表示该数据属于第一类，我们希望 $\\hat{y}$ 越小越好，上面的 loss 函数变为\n",
    "\n",
    "$$\n",
    "loss = - (log(1 - \\hat{y}))\n",
    "$$\n",
    "\n",
    "在训练模型的时候我们希望最小化 loss 函数，根据 log 函数的单调性，也就是最小化 $\\hat{y}$，与我们的要求是一致的。\n",
    "\n",
    "而如果 y 是 1，表示该数据属于第二类，我们希望 $\\hat{y}$ 越大越好，同时上面的 loss 函数变为\n",
    "\n",
    "$$\n",
    "loss = -(log(\\hat{y}))\n",
    "$$\n",
    "\n",
    "我们希望最小化 loss 函数也就是最大化 $\\hat{y}$，这也与我们的要求一致。\n",
    "\n",
    "所以通过上面的论述，说明了这么构建 loss 函数是合理的。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面我们通过例子来具体学习 Logistic 回归"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import torch\n",
    "from torch.autograd import Variable\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<torch._C.Generator at 0x1ba3948fe10>"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 设定随机种子\n",
    "torch.manual_seed(2017)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们从 data.txt 读入数据，感兴趣的同学可以打开 data.txt 文件进行查看\n",
    "\n",
    "读入数据点之后我们根据不同的 label 将数据点分为了红色和蓝色，并且画图展示出来了"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x1ba451ed880>"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAAAsTAAALEwEAmpwYAAAgF0lEQVR4nO3dfYxd9X3n8ffXT3GnOAHbk7R48HiSmgZDoMJjt1ltukhZFOcJl5YoBid0SVXL2SXKP0ugJaErIWtTVSulCWmRN0oDjBWEtklKtgQ3ArWRVpvEY2qDwcAaA2YgEsOMlIYnEZPv/nHu4Dt37sO5956H3++cz0u6Gt9zj+/93nPO/Z7f+T0dc3dERCR+y8oOQEREsqGELiJSEUroIiIVoYQuIlIRSugiIhWxoqwPXr9+vW/atKmsjxcRidLhw4dfcvfRdq+VltA3bdrE9PR0WR8vIhIlM3u202uqchERqQgldBGRilBCFxGpCCV0EZGKUEIXEamIngndzL5pZi+a2bEOr5uZfdXMTpjZw2Z2afZhZujAAdi0CZYtS/4eOFB2RCIimUhTQv8WsKPL6x8GNjcee4C/HT6snBw4AHv2wLPPgnvyd88eJXURqYSeCd3dfwTMd1llJ3CnJ34MnG1mv5lVgJm6+WZ49dXFy159NVkuIhK5LOrQNwDPNT2faSxbwsz2mNm0mU3Pzs5m8NF9OnWqv+UiIhHJIqFbm2Vt75rh7vvdfdLdJ0dH245czdfGjf0tz4iq7etJ+12KlkVCnwHOa3o+BryQwftmb98+GBlZvGxkJFmeE1Xb15P2u5Qhi4R+L3Bto7fL7wE/d/efZfC+2du9G/bvh/FxMEv+7t+fLM+Jqu3LU2YJWftdypCm2+K3gf8L/LaZzZjZn5jZXjPb21jlPuAkcAL4n8B/zi3aLOzeDc88A7/6VfI3x2QOqrYvUnMCX78ePvOZ8krIMex3VQlVkLuX8ti6davXwfi4e5JSFj/Gx8uOrFqmptxHRtpv6zK2+6D7fWoqWccs+Ts1lU987bbXyEh+nyfZAaa9Q17VSNGclVBtX0vtqjjaKaqEPMh+L7LeXVVC1VTdhB7I9WQJ1fa1lDZR59yh6S2D7Pcik2wMVULSP0tK8MWbnJz03G5wsVDUaf51jIwok1bYpk1Jibab0A+BZcuSknkrs6TJJ0udttf4eNK0JOEys8PuPtnutWqW0HU9WTvtqjhWroR16+K5MipymISqAqupmgld15O1066K4+/+Dl56qbAOTUMrMsmqKrCaqlnloutJidSBA8mF5KlTScl83z4lWVmsflUuup6USBU8TEIqppoJXdeTIlJD1UzooKKOBCuQHrVSQSvKDkCkTlp71C4MHgKVOWR41S2hiwRIPWolT0roIgWKqUetqobio4QuUqCS7rHSN83nHicldJECxdKjVlVDcVJCFylQLD1qY6oakjPUy0WkYLt3h5fAW61dC3Nz7ZdLuFRCF6kANWAKqIQuEr08+rbPz/e3XMKgErpIhJpL5H/8x9k3YMbSG0cWU0KXTOnSP3+tXQrffLP9esM0YMbSG0cWS5XQzWyHmT1hZifM7KY2r59jZt81s4fN7KdmdlH2oUro1He5GGnvnzpMaTqW3jiyWM+EbmbLga8DHwa2AFeb2ZaW1f4cOOLuFwPXAn+ddaDSnzJKyuq7XIw0Je8sStNVmN+ubleMaUro24ET7n7S3d8A7gZ2tqyzBXgAwN0fBzaZ2bsyjVRSK6ukrL7LxehU8l6+XKXpZnW8YkyT0DcAzzU9n2ksa3YU+EMAM9sOjANjrW9kZnvMbNrMpmdnZweLWHoqq6SshrRidKrfvuOOuEvTWavjFWOahG5tlrXet+7LwDlmdgT4HPCvwOkl/8l9v7tPuvvk6Ohov7FKSmWVlNWQVoxu9dt1q2Jo1vrd292FEip+xejuXR/A+4GDTc//DPizLusb8Azw9m7vu3XrVpfE1JT7+Li7WfJ3amq49xsfd08uMhc/xseHj7WXrL+LpDc15T4ysnifj4zUYx+0++5m5f0O8gRMe6f82+mFt1ZIBh+dBCaAVSTVKxe2rHM2sKrx7z8F7uz1vkroiTx+hHX+YddZmSfysnX67q1JPc/fQVGFmaESevL/+QjwJPAUcHNj2V5gr58pxf8/4HHgO8A5vd5TCT2R149QJeX66VQiNSs7svx1+u4Lv6W8fwdFFqK6JXRLXi/e5OSkT09Pl/LZIVm2LNn9rcySBi7p7MCBpIHr1Kmk4XXfvno3BnaqNx4fTxpKq6zs717k55vZYXefbPeaRoqWTD1DBlPHLmmdLDQGPvtsUhBoVpdG6bIb5EPpsquEXrKyD8RYhdglrYweJs0nNkhObgtJvU790cse2RpMwaxTXUzeD9Whn6H67v6FVl9cVkN0nRtCQ6I6dNWhyxDKrjNtVWQ8zW0HnX6+aoMpXlFtOqpDl8oJraqqqDrU1raDTtQGU7wQ5r5RQpcolV1n2qqoOtQ0My2qDaa+lNAlWiGUiBYUdcXQrcQfwolNyqVb0IlkYCGB5l2HunFjWG0HEhaV0EUyUsQVQ2htBxIWJXSRiITWdiBhUZWLSGR271YCl/ZUQhcRqQgldBGRilBCFxGpCCV0qbU637JNqkeNolJbC8PoF0ZeLkzBC2p0lDiphC61FeIUvCLDUEKX2grlpgQiWVFCl9oK5qYEIhlJldDNbIeZPWFmJ8zspjavv8PMvm9mR83sUTO7LvtQRbKlYfRSNT0TupktB74OfBjYAlxtZltaVvsvwGPufglwGfA/zGxVxrFWk7pZlEbD6KVq0pTQtwMn3P2ku78B3A3sbFnHgTVmZsBZwDxwOtNIq0h3Oi5dCFPw6pwuWUmT0DcAzzU9n2ksa3YbcAHwAvAI8Hl31w2welE3i9rTOV2ylCahW5tlrTe/+hBwBDgX+B3gNjN7+5I3MttjZtNmNj07O9tnqDkou2ikbhalK/sQ0DldspQmoc8A5zU9HyMpiTe7DvhO46bUJ4Cngfe2vpG773f3SXefHB0dHTTmbIRQNFI3i1KFcAjonC5ZSpPQDwGbzWyi0dC5C7i3ZZ1TwAcBzOxdwG8DJ7MMNHMhFI3UzaJUIRwCOqeHo+yrtSz0TOjufhq4HjgIHAfucfdHzWyvme1trHYr8O/M7BHgAeBGd38pr6AzEULRSN0sShXCIaBzehhCuFrLhLuX8ti6dauXanzcPdl3ix/j4+XGJYUJ5RCYmko+0yz5OzVV7OcPKta42wnlWEgDmPYOebW+I0XzLBpV4dqtBkIpHYfQdbJflSnRNoRwtZaJTpk+70fpJXT3fIoYU1PuIyOLT/MjI9EUX0IsdeUZU4jfNwYxlWjTiOn70KWEXu+EnoeYjowWIZ6LQoypagY5qZm1P8zN8o42HzEdZ0rorfIslkV8pId4LgoxpioZNJFVcb/EcrXWLaFb8nrxJicnfXp6uvgPbr2rASQVp1n1Ltm0KalQbDU+nlSQBmzZsuRn2cosqd8tQ4gxVcmgh2vePyPpzMwOu/tku9fq1yiad+fjUFraBhBin+gQY6qSQRsD1eM2TPVL6Hk3Z0d8pId4LgoxpirpdGJctqx3J60Ye+dUXqe6mLwfpdWhV7HyL0Mh1iOGGFNVtKtDb32E2jhYV6gOvYkq/0QWOXAgqXE8dSoplb/55tJ1ImgCqg3VoTdLUyWigUHR0K4aXnPVSaeG5ugG2NRU/UrovagEH412u2rVKlizBubnk/rhffu02/oRcSet2lAJvR8hTMEnqbTbVW+8AXNz1RiOXgY1Qucr7ytKJfRWlZnUofrS7BKdi/sTcSet4BUx/42qXFrpmjManXZVKw1CkhBklVpU5dKPrK851WqXm3a7qh0NQpIQFHHxr4TeKstrzqrNMRqY1l21bh2sXLl4HdX/SiiKGPWsKpc8qfqmcM19qtXLRUKSVQc6VbmURQ2shdNw9KVU6xeGIhqcldDzpJmlSle1ZNbv96larV/s+zP3AkenOQHyflT2BhfNYpo1P4XY5lSp2OYf6PtUaeqiqu3PQTHsDS6AHcATwAngpjav3wAcaTyOAW8Ca7u9Zy0Sunt8WbCDGH9MVUpm7oN9n4jvt7JE1fbnoIZK6MBy4Cng3cAq4Ciwpcv6Hwce7PW+tUnoFRHjj6lKycx9sO/Tab8t7LuQT8itQt6fvcptWZbruiX0NHXo24ET7n7S3d8A7gZ2dln/auDbqet8JAoxtu9WrQljkO/Tra9+bPXpoe7PXu0UhbZjdMr0Cw/gKuAbTc8/DdzWYd0RYJ4O1S3AHmAamN64cePgpygpXIwl9BiriboZ9PsslA67ldRjEOr+7PXbyPq3w5BVLp9ok9C/1mHdTwLf7/WeriqX6IT6Y+qlIk0Ybxnm+4RcZZFWiPuz13bNert3S+grUhTiZ4Dzmp6PAS90WHcXqm6ppIXuVbEN2tm9O/wY+zHM99m4sf04t7KrLPoR4v7stV2L3O5p6tAPAZvNbMLMVpEk7XtbVzKzdwD/AfiHbEOUUGjQTtw0NW4+em3XIrd7z4Tu7qeB64GDwHHgHnd/1Mz2mtneplWvBP7J3V/JPkwRGZamxs1Hr+1a5HbXXC4x0UQlwdCukLJ0m8slTR26hKB1Zp+Fvk+gTFIw7QoJleZyiYVujRcM7YowxD6vSx6U0GMR48ieitKuKF9eg3ViP0kooWehiKMg1GFyNaRdUb48rpKqMDOlEvqwijoKIu9zFnvJp1nku6IS8rhKqkRVWqcRR3k/KjNStMgx8SEOk0sh1lGm3US6Kyojj59dLCNp6TJSVN0Wh7VsWbLfW+lW82/Rnfgka1ndzq1ZLMepbkGXJ1Wo9qRGRMlaHoN1qlCVpoQ+rCocBTnTOU/ykPVUFFUYSauEPqwqHAX9GKB1U+c8iUXs8xVppGgWQpwCLg8DDpGMdaZGkdioUVTSi6XVSKTC1Cgq2VDrpkjQlNAlPbVuigRNCV3SU+umSNCU0CW9uvXoaVGl6QukOEUeN+rlIv1p7bKyMNFFxZO65kCXQRR93NS7hK4iV/+qMCXdACoxcZMUrujjpr4JvaaJqa1+Tmw1zWzq4CODKPq4qW9Cr2liWqLfE1tOR2joF0vq4CODKPq4SZXQzWyHmT1hZifM7KYO61xmZkfM7FEz+5dsw8yBilyJfk9sORyhMVwsqYOPDKLw46bTvLoLD2A58BTwbmAVcBTY0rLO2cBjwMbG83f2et/S50Mvch7zkPU7CXQOk5vHsis0B7oMIuvjhi7zoacpoW8HTrj7SXd/A7gb2NmyzjXAd9z9VOMk8eKQ55n8qciV6LfEnUPXxVgulmKfuEnKUeRxkyahbwCea3o+01jW7HzgHDP7ZzM7bGbXtnsjM9tjZtNmNj07OztYxFmpeZ/qtwxyYsv4CFX9tEg20iR0a7OsdUavFcBW4KPAh4Avmdn5S/6T+353n3T3ydHR0b6DzZyKXEGc2HSxJJKNNAOLZoDzmp6PAS+0Wecld38FeMXMfgRcAjyZSZSSr5Kn/9X0uiLZSFNCPwRsNrMJM1sF7ALubVnnH4APmNkKMxsBfhc4nm2oUmW6WKqP0LuoxqxnCd3dT5vZ9cBBkh4v33T3R81sb+P12939uJndDzwM/Ar4hrsfyzNwEYmPplDIV6p+6O5+n7uf7+7vcfd9jWW3u/vtTev8lbtvcfeL3P0rOcUroCKOBK/TIarxfPnS5FyxURFHAtftEI2li2qsdAu62Og2cBK4boco6PAdlm5BVyUq4kjguh2i6qKaLyX02GgUjgSu2yEawLCHSlNCj42KOBK4XoeouqjmRwk9NiriSOB0iJZHjaIiIhFRo6iISA0ooYuIVIQSuohIRSihi+RAszNIGZTQJT81zWox3CNVqkkJva7yTrahZLUSTiqagErKom6LddQ6exIkIz+y7CwcwpwzRXzPNpYtS85hrcySwTQiw+jWbVEJvY6KSLYhZLWSTiohnMukutQPXRYrYoKvEOacKWkiM83OIGVRQq+jIpJtCFmtpJOKhr5LWZTQ66iIZFt2VjtwAF5+eenygk4qmoBKyqCEXkfdkm2WvULKymoLjaFzc4uXr1unorJUWqqEbmY7zOwJMzthZje1ef0yM/u5mR1pPG7JPlRq2685F+2SbShdDYfVrt8gwFlnKZlLpfXs5WJmy4EngcuBGeAQcLW7P9a0zmXAf3X3j6X94L57uZTUBa1WqtI9I4QeNiI5GbaXy3bghLufdPc3gLuBnVkGmIpGa+SvU++Pdkk+ZCH0sBEpQZqEvgF4run5TGNZq/eb2VEz+4GZXdjujcxsj5lNm9n07Oxsf5HqXpr565TwzOKqdgmhh02NqWa0PGkSurVZ1no9+xAw7u6XAF8Dvtfujdx9v7tPuvvk6OhoX4Gq1FWAffuS5N3KPa4robJ72NRYVZphYpUmoc8A5zU9HwNeaF7B3f/N3V9u/Ps+YKWZrc8sSlCpqwi7d7eve4b4roTUb7AUqhktV5qEfgjYbGYTZrYK2AXc27yCmf2GWVK0M7PtjfedW/JOw1Cpqxjj4+2X60ooSkVXf6hmtFw9E7q7nwauBw4Cx4F73P1RM9trZnsbq10FHDOzo8BXgV2exyQxeZa6VPGX0JVQZZRR/aGa0ZK5eymPrVu3ejCmptxHRtyT4z55jIwky+toasp9fNzdLPn72c8ufl7X7dJJ6/YKZPuMjy8+pBce4+P5faZ+SvkDpr1DXlVCdy/nyI+FfqHdlbx9up1LzNof1mblxSTD65bQNX0uaCBKN1UZbJSXErdPr7F22nXVpOlze1HFX3sHDnQeVKRWrkSJrYC9epSoOWRwsTapKaGDjvx2Fop/ndT9ZLegxMJAr3NJHTqG5ZF4o+5L36kuJu9HUHXo7qr4a9WpXUF16IuVWIde96afvDZ96NsVNYpK3zq1qIGSeauSCgN5nktiKN/klXjLakxOq1tCV6OotKcWtSgcOJDUmZ86ldTy7Ns3fJVKLBOb5tWXIfRDv1qNollVmsXa6lEUtStEIY+xdrEM38+r+SLqQ79T0T3vx0BVLlldY6pvdToxXHdL5rKucsjrMKprlROVqUPvt9Ks014JvdVDpERZ/jzyLjs1/8TXrUseISbhLFUnofdTdOh2JIXe6iHhC7kIN6Qsk3BRZac6XXRXJ6H3c3R0W1cl9OooI7HWIHuk2qwpViqq7FSnn3R1Eno/P6RuR1INfpC1UNZ+rFP26CTlti9qU9Xpors6Cd09fYms15FU4Uvm2igrsdYpe3SSctsXdc6t0zm2Wgk9LZXCq6+sxBpK9iizUNLHti8izDr93OuZ0N1VCq+6shJrCNmj7BhCOak1qcvPvb4JXaqt6KQWUh+5shNq2SeUGuuW0FeUNqJJZFgLwyKzHvveTut4+Lm5ZPjgXXeVMx6+7Jt3Lnznz38+2RYAv/ZrxXy2dBTf0H+RZnneZ7ZZaOPhQ5nD/7XXzvx7bi6ieWarKajJuX75y18yMzPD66+/XkpMWVu9ejVjY2OsXLmy7FBkWKHd1SqEGbRCn8WqorpNzpWqysXMdgB/DSwHvuHuX+6w3jbgx8An3f1/9RvozMwMa9asYdOmTZhZv/89KO7O3NwcMzMzTExMlB2ODGvjxvbJq6wbfRRZ3dRJ2dU+skTPKhczWw58HfgwsAW42sy2dFjvL4GDgwbz+uuvs27duuiTOYCZsW7duspcbWQuttkuQ5yCr6jqpk5CqfaRt6SpQ98OnHD3k+7+BnA3sLPNep8D/h54cZiAqpDMF1Tpu2Qqxnt81eF+bv0K8SSXVmwFipTSJPQNwHNNz2cay95iZhuAK4Hbu72Rme0xs2kzm56dne03VqmK0BoY0yq7RByaWE9yMRYoUkqT0NsVM1tbh74C3Ojub3Z7I3ff7+6T7j45OjqaMsQuKnqWrTzVvVZHjCe5QQoUkeSaNAl9Bjiv6fkY8ELLOpPA3Wb2DHAV8Ddm9gdZBNhRyWfZO+64g82bN7N582buuOOOQj6zMlT3KmXqt0ARU4m+04ijhQdJT5iTwASwCjgKXNhl/W8BV/V633YjRR977LH0w6VKHCk3NzfnExMTPjc35/Pz8z4xMeHz8/Nt1+3rO9WFRhlKmfrNHWWPym1Bl5GiPUvo7n4auJ6k98px4B53f9TM9prZ3jxOMqnkcNl+6NAhLr74Yl5//XVeeeUVLrzwQo4dO7ZkvYMHD3L55Zezdu1azjnnHC6//HLuv//+gT+3dmKte5Vq6LcxN6IqwlT90N39PuC+lmVtG0Dd/T8NH1YKOfQL3rZtG1dccQVf/OIXee211/jUpz7FRRddtGS9559/nvPOO1MLNTY2xvPPPz/w59bS7t1K4FKOfvvwhzYGoYt4h/7n1GXqlltu4Yc//CHT09N84QtfaLuOtxkxqC6KIhHppzE3ou6Z8Sb0nC7b5+fnefnll/nFL37RcVDQ2NgYzz13pifnzMwM55577lCfKyKBiqiKMKi5XI4fP84FF1xQSjwLrrjiCnbt2sXTTz/Nz372M2677bYl68zPz7N161YeeughAC699FIOHz7M2rVrl6wbwncSkeroNpdLvCX0HNx5552sWLGCa665hptuuolDhw7x4IMPLllv7dq1fOlLX2Lbtm1s27aNW265pW0ylyaR9OMViZlK6Dmr4nfqWwgzA4pUhEroUq5Yh/qLREZ3LOrikUce4dOf/vSiZW9729v4yU9+UlJEkYqoH69IzJTQu3jf+97HkSNHyg4jfhH14xWJmapcJH8R9eMViZkSuuQvon68IjFTlYsUQ0P9RXIXdQldXZtFRM6INqGXPUXxjh07OPvss/nYxz5WzAeKiPQQbUIvu2vzDTfcwF133VXMh4mIpBBtQs+ja3Pa+dABPvjBD7JmzZrBP0xEJGPRNorm0bU57XzoIiIhiraEnlfX5jTzoYuIhCjahJ5X1+Y086GLBENdvaRJtFUukE/X5j179nDrrbfy9NNPc+ONN7adD10kCK2zWC509QL1+a+pVCV0M9thZk+Y2Qkzu6nN6zvN7GEzO2Jm02b277MPNX9p50MH+MAHPsAnPvEJHnjgAcbGxjh48GDB0Urtld3VS4LTcz50M1sOPAlcDswAh4Cr3f2xpnXOAl5xdzezi4F73P293d5X86GLDGnZsmQQRiuz5F6ZUknDzoe+HTjh7ifd/Q3gbmBn8wru/rKfOTP8OlDOXTNE6qRTly7NYllbaerQNwDPNT2fAX63dSUzuxL478A7gY9mEl3JNB+6BG3fvvZ3gtIslrWVJqFbm2VLSuDu/l3gu2b2+8CtwH9c8kZme4A9ABs7lCLcHbN2H1m8YedDL+v2flITCw2fN9+cjKjbuDFJ5moQra00VS4zwHlNz8eAFzqt7O4/At5jZuvbvLbf3SfdfXJ0dHTJ/129ejVzc3OVSITuztzcHKtXry47FKmy3bvhmWeSOvNnnlEyr7k0JfRDwGYzmwCeB3YB1zSvYGa/BTzVaBS9FFgFzPUbzNjYGDMzM8zOzvb7X4O0evVqxsbGyg5DRGqiZ0J399Nmdj1wEFgOfNPdHzWzvY3Xbwf+CLjWzH4JvAZ80gcoZq9cuZKJiYl+/5uIiJCi22Je2nVbFBGR7obttigiIhFQQhcRqYjSqlzMbBZoMwFuKuuBlzIMJ08xxQqKN08xxQpxxRtTrDBcvOPuvrSbICUm9GGY2XSnOqTQxBQrKN48xRQrxBVvTLFCfvGqykVEpCKU0EVEKiLWhL6/7AD6EFOsoHjzFFOsEFe8McUKOcUbZR26iIgsFWsJXUREWiihi4hURNAJPaZb3/WKtWm9bWb2ppldVWR8beLotW0vM7OfN7btETO7pYw4G7H03LaNeI+Y2aNm9i9Fx9gSS69te0PTdj3WOB7WBhrrO8zs+2Z2tLFtrysjzqZ4esV7jpl9t5EXfmpmF5URZyOWb5rZi2Z2rMPrZmZfbXyXhxsTGw7H3YN8kEwE9hTwbpLZG48CW1rWOYsz7QAXA4+HGmvTeg8C9wFXBb5tLwP+dyTHwdnAY8DGxvN3hhxvy/ofBx4MNVbgz4G/bPx7FJgHVgUc718Bf9H493uBB0o8Fn4fuBQ41uH1jwA/ILnnxO8BPxn2M0Muocd067uesTZ8Dvh74MUig2sjbbwhSBPrNcB33P0UgLuXuX373bZXA98uJLKl0sTqwBpL7jpzFklCP11smG9JE+8W4AEAd38c2GRm7yo2zIQn94aY77LKTuBOT/wYONvMfnOYzww5obe79d2G1pXM7Eozexz4R+AzBcXWqmesZrYBuBK4vcC4Okm1bYH3Ny61f2BmFxYT2hJpYj0fOMfM/tnMDpvZtYVFt1TabYuZjQA7SE7yZUgT623ABSQ3tXkE+Ly7l3UH6jTxHgX+EMDMtgPjJDflCVHqYyWtkBN66lvfuft7gT8gufVdGdLE+hXgRnd/M/9wekoT70Mkc0ZcAnwN+F7eQXWQJtYVwFaSe9l+CPiSmZ2fd2AdpDpuGz4O/B9371aKy1OaWD8EHAHOBX4HuM3M3p5vWB2liffLJCf3IyRXxP9KeVcUvfRzrKSS5o5FZen71ndm9h4zW+/uRU/SkybWSeDuxv1S1wMfMbPT7v69QiJcrGe87v5vTf++z8z+JuBtOwO85O6vAK+Y2Y+AS4AniwlxSSxpj9tdlFfdAulivQ74cqNq84SZPU1SN/3TYkJcJO1xex0kjY7A041HiPrKcamU1WCQokFhBXASmOBMA8iFLev8FmcaRS8luUWehRhry/rfotxG0TTb9jeatu124FSo25akSuCBxrojwDHgolC3bWO9d5DUr/564MfB3wL/rfHvdzV+Y+sDjvdsGo22wJ+S1FGXsn0bMWyic6PoR1ncKPrTYT8v2BK6F3jru4JiDUbKeK8CPmtmp0m27a5Qt627Hzez+4GHgV8B33D3tl3FQoi3seqVwD95clVRipSx3gp8y8weIUk8N3rxV2n9xHsBcKeZvUnS8+lPyogVwMy+TdJbbL2ZzQB/AayEt2K9j6SnywngVRpXFkN9Zgm/URERyUHIjaIiItIHJXQRkYpQQhcRqQgldBGRilBCFxGpCCV0EZGKUEIXEamI/w+U6Wmm4uuHBQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 从 data.txt 中读入点\n",
    "with open('./data.txt', 'r') as f:\n",
    "    data_list = [i.split('\\n')[0].split(',') for i in f.readlines()]\n",
    "    data = [(float(i[0]), float(i[1]), float(i[2])) for i in data_list]\n",
    "\n",
    "# 标准化\n",
    "x0_max = max([i[0] for i in data])\n",
    "x1_max = max([i[1] for i in data])\n",
    "data = [(i[0]/x0_max, i[1]/x1_max, i[2]) for i in data]\n",
    "\n",
    "x0 = list(filter(lambda x: x[-1] == 0.0, data)) # 选择第一类的点\n",
    "x1 = list(filter(lambda x: x[-1] == 1.0, data)) # 选择第二类的点\n",
    "\n",
    "plot_x0 = [i[0] for i in x0]\n",
    "plot_y0 = [i[1] for i in x0]\n",
    "plot_x1 = [i[0] for i in x1]\n",
    "plot_y1 = [i[1] for i in x1]\n",
    "\n",
    "plt.plot(plot_x0, plot_y0, 'ro', label='x_0')\n",
    "plt.plot(plot_x1, plot_y1, 'bo', label='x_1')\n",
    "plt.legend(loc='best')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 对该部分进行重写\n",
    "import pandas as pd\n",
    "dfData = pd.read_csv(\"./data.txt\",sep = ',',header=None)\n",
    "dfData.columns = [\"x1\",\"x2\",\"label\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Max-Min-Scale\n",
    "def MMScale(x):\n",
    "    return (x-x.min())/(x.max()-x.min())\n",
    "\n",
    "dfData[\"x1\"] = MMScale(dfData[\"x1\"])\n",
    "dfData[\"x2\"] = MMScale(dfData[\"x2\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>x1</th>\n",
       "      <th>x2</th>\n",
       "      <th>label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.065428</td>\n",
       "      <td>0.694655</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.003266</td>\n",
       "      <td>0.194705</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.082968</td>\n",
       "      <td>0.619618</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.215346</td>\n",
       "      <td>0.376660</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>0.943151</td>\n",
       "      <td>0.111651</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>0.644338</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>0.135870</td>\n",
       "      <td>0.665535</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>0.543049</td>\n",
       "      <td>0.235480</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>0.534810</td>\n",
       "      <td>0.179227</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>0.293482</td>\n",
       "      <td>0.267373</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>0.059528</td>\n",
       "      <td>0.199312</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>0.903776</td>\n",
       "      <td>0.120080</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>0.455379</td>\n",
       "      <td>0.287886</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>0.125084</td>\n",
       "      <td>0.503799</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>0.316031</td>\n",
       "      <td>0.476435</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>0.145882</td>\n",
       "      <td>0.594211</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>0.352252</td>\n",
       "      <td>0.316564</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>0.055278</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>0.641122</td>\n",
       "      <td>0.160697</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>0.059121</td>\n",
       "      <td>0.653830</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>0.308001</td>\n",
       "      <td>0.238083</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>43</th>\n",
       "      <td>0.749759</td>\n",
       "      <td>0.146705</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>0.300834</td>\n",
       "      <td>0.222943</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>45</th>\n",
       "      <td>0.461005</td>\n",
       "      <td>0.314324</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>53</th>\n",
       "      <td>0.064007</td>\n",
       "      <td>0.436425</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>54</th>\n",
       "      <td>0.289923</td>\n",
       "      <td>0.281271</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>55</th>\n",
       "      <td>0.279893</td>\n",
       "      <td>0.427821</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>57</th>\n",
       "      <td>0.036096</td>\n",
       "      <td>0.952086</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>61</th>\n",
       "      <td>0.074923</td>\n",
       "      <td>0.240489</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>62</th>\n",
       "      <td>0.375453</td>\n",
       "      <td>0.126830</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>63</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.278172</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>64</th>\n",
       "      <td>0.209397</td>\n",
       "      <td>0.525104</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>65</th>\n",
       "      <td>0.523184</td>\n",
       "      <td>0.153646</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>67</th>\n",
       "      <td>0.272524</td>\n",
       "      <td>0.311720</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>70</th>\n",
       "      <td>0.038183</td>\n",
       "      <td>0.186094</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>78</th>\n",
       "      <td>0.435677</td>\n",
       "      <td>0.174393</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79</th>\n",
       "      <td>0.747722</td>\n",
       "      <td>0.177491</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>86</th>\n",
       "      <td>0.172234</td>\n",
       "      <td>0.706668</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>89</th>\n",
       "      <td>0.319471</td>\n",
       "      <td>0.441892</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>92</th>\n",
       "      <td>0.364393</td>\n",
       "      <td>0.072766</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          x1        x2  label\n",
       "0   0.065428  0.694655      0\n",
       "1   0.003266  0.194705      0\n",
       "2   0.082968  0.619618      0\n",
       "5   0.215346  0.376660      0\n",
       "10  0.943151  0.111651      0\n",
       "11  0.644338  0.000000      0\n",
       "14  0.135870  0.665535      0\n",
       "17  0.543049  0.235480      0\n",
       "20  0.534810  0.179227      0\n",
       "22  0.293482  0.267373      0\n",
       "23  0.059528  0.199312      0\n",
       "27  0.903776  0.120080      0\n",
       "28  0.455379  0.287886      0\n",
       "29  0.125084  0.503799      0\n",
       "32  0.316031  0.476435      0\n",
       "34  0.145882  0.594211      0\n",
       "35  0.352252  0.316564      0\n",
       "36  0.055278  1.000000      0\n",
       "38  0.641122  0.160697      0\n",
       "39  0.059121  0.653830      0\n",
       "41  0.308001  0.238083      0\n",
       "43  0.749759  0.146705      0\n",
       "44  0.300834  0.222943      0\n",
       "45  0.461005  0.314324      0\n",
       "53  0.064007  0.436425      0\n",
       "54  0.289923  0.281271      0\n",
       "55  0.279893  0.427821      0\n",
       "57  0.036096  0.952086      0\n",
       "61  0.074923  0.240489      0\n",
       "62  0.375453  0.126830      0\n",
       "63  0.000000  0.278172      0\n",
       "64  0.209397  0.525104      0\n",
       "65  0.523184  0.153646      0\n",
       "67  0.272524  0.311720      0\n",
       "70  0.038183  0.186094      0\n",
       "78  0.435677  0.174393      0\n",
       "79  0.747722  0.177491      0\n",
       "86  0.172234  0.706668      0\n",
       "89  0.319471  0.441892      0\n",
       "92  0.364393  0.072766      0"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# for i in dfData[dfData['label']==0]:\n",
    "#     print(i)\n",
    "dfData[dfData['label']==0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x1ba451c4d90>"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAAAsTAAALEwEAmpwYAAAcvklEQVR4nO3dfYwkV3nv8e+zbywjNrF3dhIFj2dmEy3ENpjIM8slKCTkWhZrB9mKhCXjxcslkVariIj8EbATg68sa6XwH0lMgvZagO0ZYUUJSpwrrlfIKBckgtlZ5Fcs0LKO17O25N1ZK8EvK7/w5I/qsXt6+qW6u6rOOVW/j9RaT3e5+1RX9VNPPefUKXN3REQkfZtCN0BERIqhgC4iUhMK6CIiNaGALiJSEwroIiI1sSXUB+/atcvn5uZCfbyISJKOHz9+1t2nur0WLKDPzc2xvLwc6uNFRJJkZk/3ek0lFxGRmlBAFxGpCQV0EZGaUEAXEakJBXQRkZoYGNDN7Ktm9ryZPd7jdTOzvzGzE2b2qJldUXwzC7S0BHNzsGlT9u/SUugWiYgUIk+G/nVgX5/Xrwb2tB4Hgb8fv1klWVqCgwfh6afBPfv34EEFdRGphYEB3d2/C5zrs8h1wD2e+QFwgZn9WlENLNStt8LLL69/7uWXs+dFRBJXRA39IuCZtr9XWs9tYGYHzWzZzJbPnDlTwEcP6dSp4Z4XEUlIEQHdujzX9a4Z7n7E3RfcfWFqquuVq+WamRnu+YKobN9M2u5StSIC+gpwcdvf08CzBbxv8Q4fhomJ9c9NTGTPl0Rl+2bSdpcQigjo9wMHWqNdPgD8p7s/V8D7Fm//fjhyBGZnwSz798iR7PmSqGwfTsgMWdtdQsgzbPEbwL8D7zazFTP7YzM7ZGaHWot8CzgJnAD+D/AnpbW2CPv3w3/8B/ziF9m/JQZzUNm+Su0BfNcu+KM/Cpchp7DdVRKqIXcP8pifn/cmmJ11z0LK+sfsbOiW1cviovvERPfvOsT3Pup2X1zMljHL/l1cLKd93b6viYnyPk+KAyx7j7iqK0VLFqBs30jdShzdVJUhj7Ldq6y7qyRUT/UN6JGcTwYo2zdS3kBd8oCmN42y3asMsimUhGR4lmXw1VtYWPDSbnCxluq0/zomJhRJa2xuLsto+4l9F9i0KcvMO5llXT5F6vV9zc5mXUsSLzM77u4L3V6rZ4au88nG6Vbi2LoVJifTOTOq8jIJlQLrqZ4BXeeTjdOtxPG1r8HZs5UNaBpblUFWpcB6qmfJReeTkqilpexE8tSpLDM/fFhBVtZrXslF55OSqIovk5CaqWdA1/mkiDRQPQM6KNWRaEUyolZqaEvoBog0SeeI2rWLh0A5h4yvvhm6SIQ0olbKpIAuUqGURtSqNJQeBXSRCgW6x8rQNJ97mhTQRSqUyohalYbSpIAuUqFURtSmVBqSt2iUi0jF9u+PL4B32rkTVle7Py/xUoYuUgPqwBRQhi6SvDLGtp87N9zzEgdl6CIJas/IP/nJ4jswUxmNI+spoEuhdOpfvs4hhW+80X25cTowUxmNI+spoEthNHa5GnnvnzpONp3KaBxZTwG9pkJkyhq7XI08mXcR2XQd5rdr2hmjAnoNhcqUNXa5Gr0y782blU23a+IZowJ6DYXKlNWRVo1e9e277047my5aE88YFdBrKFSmrI60avSrbzetxNCuc9273YUSan7G6O5BHvPz8y6ZxUX32Vl3s+zfxcXx3m921j07yVz/mJ0dv62DFL0ukt/iovvExPptPjHRjG3Qbd3Nwv0OygQse4+4qoAeWBk/wib/sJss5IE8tF7r3hnUy/wdVJXMKKBHrKwfoTLl5umVkZqFbln5eq372m+p7N9BlUlUv4Bu2evVW1hY8OXl5SCfHZNNm7LN38ks6+CS3paWsg6uU6eyjtfDh5vdGdirbjw7m3WU1lnoda/y883suLsvdHtNnaKBaWTIaJo4JK2Xtc7Ap5/OEoF2TemUDt0hH8uQXQX0wELviKmKcUhaiBEm7Qc2yA5ua0G9SePRQ1/ZGk1i1qsW0/4A9gE/AU4At3R5/ZeBfwUeAZ4APjXoPVVDf4vq3cOLrV4cqiO6yR2hMYmlhp4nmG8Gfgb8OrCtFbQv7VjmL4Evtv57CjgHbOv3vgroMo7YAlmV7WlPAHp1BDahIzQ2MYxyyVNyeT9wwt1PuvurwH3AdZ2JPrDDzAx4Ryugvz78+YJIPrGVqqqqoXb2HfSiPpjqxTD3TZ6AfhHwTNvfK63n2t0JXAI8CzwGfMbdN4zRMLODZrZsZstnzpwZscki4WumnaqqoeaZaVF9MM2VJ6Bbl+c6c4OPAA8D7wR+C7jTzH5pw//kfsTdF9x9YWpqasimiqwXQ0a0pqozhn4ZfwwHNgkrT0BfAS5u+3uaLBNv9yngm60SzwngKeA3i2miSPyqOmPolfHPzsZxYJOw8gT0Y8AeM9ttZtuAG4D7O5Y5BVwJYGa/CrwbOFlkQ0ViV8UZQ2x9BxKXgQHd3V8HPg0cBZ4E/sHdnzCzQ2Z2qLXYHcAHzewx4EHgZnc/W1ajRZoqtr4DiYsu/RcRSYgu/RcRaQAFdBGRmlBAFxGpCQV0abQm37JN6mdL6AaIhLJ2Gf3alZdrU/CCRo1ImpShS2PFOAWvyDgU0KWxYrkpgUhRFNClsaK5KYFIQRTQpbF0Gb3UjQJ6aBpmEYwuo5e6UUAPSXc6Di6GKXh1TJeiKKCHpGEWjadjuhSp2QE9dGqkYRbBhd4FdEyXIjX3wqIYriqZmck+t9vzUroYdgEd06VIzc3QY0iNNMwiqBh2AQ2djEfos7UiNDegx5AaaZhFUDHsAjqmx6EufRnNDeixpEYxDLNoqBh2gZSP6XXIaNfEcLZWhOYG9DJTozrt6TUWS3ac4jG9LhntmhjO1grh7kEe8/PzHtziovvsrLtZ9u/iYjHvOTHhnu3n2WNiopj3rkAZX8m4ymxTjOubgtnZ9bv42mN2NnTLRpPS+gDL3iOuNjuglyGlPaNDjMeiGNtUN6Mc1My67+ZmZbe2HCntZwroncpMyxLe02M8FsXYpjoZNZDVcbukcrbWL6Bb9nr1FhYWfHl5ufoP7hx8DFnhtKieqLm57mPLZ2ezAmnENm3KfpadzLL6bggxtqlORt1dy/4ZSW9mdtzdF7q91rxO0bK7s2PpaRtBDKM+8n62xmkXY9TOwJRH59RZ8wJ62d3ZCe/pMR6LYmxTnfQ6MG7aNHiQVoqjc2qvVy2m7EewGnodi38FirGOGGOb6qJbDb3zEWvnYFOhGnobFf9E1llayiqOp05lWfkbb2xcJoEuoMZQDb1dnpKILgxKhjbV+NpLJ706mpO7wKahmpehD6IMPhndNtW2bbBjB5w7l9WHDx/WZhtGwoO0GkMZ+jDqMqlDA3TbVK++Cqur9bgcPQR1Qper7DNKBfROtZnUof7ybBIdi4eT8CCt6FUx/02ukouZ7QP+GtgM3OXuf9VlmQ8DXwK2Amfd/ff6vWe0JRedcyaj16bqpIuQJAZFhZaxSi5mthn4MnA1cCnwcTO7tGOZC4C/A65198uA6/M3LzJFn3Oq16403TZVN7oISWJQxcl/npLL+4ET7n7S3V8F7gOu61jmRuCb7n4KwN2fL66JFSvynLNuc4xGpnNTTU7C1q3rl1H9V2JRxVXPeQL6RcAzbX+vtJ5r9y7gQjP7NzM7bmYHur2RmR00s2UzWz5z5sxoLa5CUZfAqYO1dO2b6uxZ+NrXVP+VOFXR4ZwnoFuX5zoL71uAeeAPgI8AXzCzd234n9yPuPuCuy9MTU0N3djkqIO1crocfSNV/eJQRYdznoC+Alzc9vc08GyXZR5w95fc/SzwXeB9xTQxYZpZKri6BbNh16duVb/Ut2fpCUevOQHWHmTZ90lgN7ANeAS4rGOZS4AHW8tOAI8D7+n3vrW9wUW7lGbNzyG1OVVq9vWPtD51mrqobttzVIx7gwvgGuCnwM+AW1vPHQIOtS3zWeDHrWD+Z4PesxEB3T29KNhDij+mOgUz99HWJ+H7rWxQt+05qrEDehmPxgT0mkjxx1SnYOY+2vr02m5r2y7mA3KnmLfnoLytyLyuX0DXlaKSS4r9u3XrwhhlffqN1U+tnh7r9hzUT1FpP0avSF/2Qxl6WlLM0FMsE/Uz6vqsZYf9MvUUxLo9B/02iv7toJKLjCvWH9MgNenCeNM46xNzySKvGLfnoO+16O+9X0DX9LmSW/uNEDQ1bXo0TVE5Bn2vRX/vmj5XCqGLdtKmqXHLMeh7rfJ7V0AXaQhNjVuOQd9rld+7Si4pUc0jGtoUEkq/ksuWqhsjI+q839ra2CdQJKmYNoXESiWXVGjmxmhoU8Qh9XldyqCAnooUr+ypKW2K8Mq6WCf1g4QCehGq2AtivUyugbQpwivjLKkOM1MqoI+rqr0g8TFnqWc+7RLfFLVQxllSLUppva44KvtRmytFq7wmPsbL5HJI9SrTfhLdFLVRxs8ulStp0ZWiJdq0KdvunXSr+TfpCkUpWudII8jOksYZ353KfqorRcukgupA6kSUopVxsU4dSmkK6OOqw15QMh3zpAxFT0VRhytpFdDHVYe9YBgj9G7qmCepSH2+Il0pWoT9+9Pb8qMY8RLJtZd0qbxIudQpKvml0mskUmPqFJViqHdTJGoK6JKfejdFoqaALvmpd1Mkagrokl/TRvR0qNP0BVKdKvcbjXKR4XQOWVmb6KLmQV1zoMsoqt5vmp2hK+UaXh2mpBtBLSZukspVvd80N6A3NDB1NcyBraGRTQN8ZBRV7zfNDegNDUwbDHtgK2kPjf1kSQN8ZBRV7zfNDehKuTLDHthK2ENTOFnSAB8ZRdX7TXMDulKuzLAHthL20BROlho+wEdGVPV+09xL/8uYUDlFo1zOv7RU6MQsmlJeJD9d+t+NUq7MKBl3wVPS6WRJpBi5ArqZ7TOzn5jZCTO7pc9ye83sDTP7WHFNLFHqc2UWIYIDm+rTIsUYeGGRmW0GvgxcBawAx8zsfnf/cZflvggcLaOhUqLA0/9qel2RYuTJ0N8PnHD3k+7+KnAfcF2X5f4U+Cfg+QLbJw2hk6XmiH2IasryBPSLgGfa/l5pPfcmM7sI+EPgK/3eyMwOmtmymS2fOXNm2LaKSOJSGKKasjwB3bo81zkm4UvAze7+Rr83cvcj7r7g7gtTU1M5mygbKMWRyPXaRVMYopqyPJNzrQAXt/09DTzbscwCcJ+ZAewCrjGz1939n4topLTRLFESuX67qK7nK9fAcehmtgX4KXAlcBo4Btzo7k/0WP7rwP9193/s977Bx6GnSreBk8j120VBu++4xhqH7u6vA58mG73yJPAP7v6EmR0ys0PFNlUGUoojkeu3i2qIarlyzYfu7t8CvtXxXNcOUHf/X+M3S3qameme4ugqHIlEv11UQ1TL1dwrRVOlFEciN2gX1RDV8iigpyaCKztF+tEuGk5zJ+cSEUmQJucSEWkABXQRkZpQQBcRqQkFdJESaHYGCUEBXcrT0KimCagkFAX0pio72MYS1QIcVDQBlYSiYYtNVMX9VGOYcybQfWN1j1QpU79hiwroTVRFsI0hqgU6qMRwLJP60jh0Wa+KCb5iuPNzoInMNDuDhKKA3kRVBNsYolqgg4oufZdQFNCbqIpgGzqqLS3Biy9ufL6ig4omoJIQFNCbqF+wLXJUSKiottYZurq6/vnJSaXKUmtpBfSGjmsuRbdgG8tQw3F1GzcI8I53KJhLraUzyiXQELRGqcvwjBhG2IiUpB6jXHS1Rvl6jf7oFuRjFsMIG5EA0gnoupdm+XoFPLO0yi4xjLBpMFVGw0knoCvrKt/hw1nw7uSe1plQ6BE2DVaXbphUqYYu63UL6GvPq/4sA9SlGyZm9aihK+uqxuxs9+d1JpSkqssfqoyGlU5Ah3LHNavwl1H9uTZClD9UGQ0rrYBeFhX+3tLtTOiTn8xq6E0/2PUSaTIQYmCY8oHA3D3IY35+3qMxO+uehfL1j9nZ0C0Lb3HRfWJi/fcyMZE9L8G/n8XFbDc1y/5t/1iz7ru1Wbg2yfiAZe8RV9PpFC2TLkTpTb1c/QX8fgaNE9Cmq6d6dIqWSYW/7paWel9UpF6uTMBewEElFZU/RhdpFW0gBXTQnt/NWvrXS9MPdmsCJgODjiVNGBhWRuBNukutVy2m7EdUNXR3Ff469epXUA19vYA19KZ3/ZT11cf+vdKnhq6ALt316lEDBfNOgZKBMo8lKeQ3ZQXeUJ3JeY0d0IF9wE+AE8AtXV7fDzzaenwfeN+g91RAj1zsaYq4ezmBN5WBTWUF3th3/X4BfWAN3cw2A18GrgYuBT5uZpd2LPYU8HvufjlwB3Bk/GJQD0UVzVLt9aiK+hWSUMa1dqlMbFpW90XSu36vSL/2AH4bONr2918Af9Fn+QuB04Ped6QMvajUIZUUJLQUzrulcEVnvmXtRk0tOTFOyQX4GHBX2983AXf2Wf7P25fveO0gsAwsz8zMDL8mw54L9doqsZ9TiQRU5M+j7Nyp/Sc+OZk9YgzCRRo3oF/fJaD/bY9lfx94Epgc9L4jZejDpA799qTYez0kfjGncGMqMghXlTs16aR73ICeq+QCXA78DHjXoPf0UQP6MHtHv2WVoddHiMDagOiR62vNsVBVuVOTftLjBvQtwElgN7ANeAS4rGOZGbIRMB8c9H5rj9Jr6P32pAb8IBsh1HZsUvToJed3X9VX1aST7rECevb/cw3w01YGfmvruUPAodZ/3wW8ADzcevT8wLXHyMMW82Zkg/akGp8yN0aowNqk6NFLzu++qmNuk46xYwf0Mh6lj0NXFl5/oQJrLNEjZFIyxHdfRTOb9HNvZkB3VxZed6ECawzRI3QbYjmotWnKz725AV3qreqgFtMYudABNfQBpcH6BfQto1yMJBKFtcsib701m2JwZia7nK+M6QQ7Jx9fXc0uH7z33jDTF4a+eefaOn/mM9l3AfD2t1fz2dKTps+VtJV5n9l2sV0PH8sc/q+88tZ/r64mNM9sPUV1x6LXXnuNlZUVzp8/H6RNRdu+fTvT09Ns3bo1dFNkXLHd1WrQ7YqqoFsiBdHvjkVRlVxWVlbYsWMHc3NzmFno5ozF3VldXWVlZYXdu3eHbo6Ma2ame/AKdaOPKstNvYQu+8gGUZVczp8/z+TkZPLBHMDMmJycrM3ZRuFSm+0yxin4qio39RJL2UfeFFVAB2oRzNfUaV0KleI9vppwP7dhxXiQyyu1hCKn6AK6NEBsHYx5hc6IY5PqQS7FhCKntAN6TY+ytafaa32keJAbJaFIJNakG9ADH2Xvvvtu9uzZw549e7j77rsr+czaUO1VQho2oUgoo083oAc8bT937hy33347Dz30ED/84Q+5/fbbeeGFF0r/3NpIufYq6Rs2oUioRJhuQC/htP3YsWNcfvnlnD9/npdeeonLLruMxx9/fMNyR48e5aqrrmLnzp1ceOGFXHXVVTzwwAMjf27jpFp7lXoYNqFIqEQY1Tj0oZQwLnjv3r1ce+21fP7zn+eVV17hE5/4BO95z3s2LHf69GkuvvjiN/+enp7m9OnTI39uI+3frwAuYQw7hj+2axD6SDdDL+m0/bbbbuPb3/42y8vLfO5zn+u6TLerazVEUSQhw3TmJlQiTDegl3Tafu7cOV588UV+/vOf97woaHp6mmeeeebNv1dWVnjnO9851ueKSKQSKhFGNZfLk08+ySWXXBKkPWuuvfZabrjhBp566imee+457rzzzg3LnDt3jvn5eX70ox8BcMUVV3D8+HF27ty5YdkY1klE6qPfXC7pZugluOeee9iyZQs33ngjt9xyC8eOHeM73/nOhuV27tzJF77wBfbu3cvevXu57bbbugZzaZPIOF6RlClDL1kd12loMcwMKFITytAlrITG8YqkLN1hixV47LHHuOmmm9Y997a3vY2HHnooUIsSldA4XpGUKaD38d73vpeHH344dDPSl9A4XpGUqeQi5UtoHK9IyhTQpXwJjeMVSZlKLlINXeovUrqkM3QNbRYReUuyAT30FMX79u3jggsu4KMf/Wg1HygiMkCyAT300ObPfvaz3HvvvdV8mIhIDskG9DKGNuedDx3gyiuvZMeOHaN/mIhIwZLtFC1jaHPe+dBFRGKUbIZe1tDmPPOhi4jEKFdAN7N9ZvYTMzthZrd0ed3M7G9arz9qZlcU39T1yhranGc+dJFoaKiXtBkY0M1sM/Bl4GrgUuDjZnZpx2JXA3taj4PA3xfczq6GuelIXgcPHuSOO+5g//793HzzzeO/oUhZQg/1kujkydDfD5xw95Pu/ipwH3BdxzLXAfd45gfABWb2awW3tXR550MH+NCHPsT111/Pgw8+yPT0NEePHq24tdJ4oYd6SXTydIpeBDzT9vcK8D9yLHMR8Fz7QmZ2kCyDZybCiZkOHDjAgQMHANi8eXPfWRW/973vVdUske40i6V0yJOhd7v7ceddMfIsg7sfcfcFd1+YmprK0z4R6aVXUhRhsiTVyJOhrwAXt/09DTw7wjLJ0XzoErXDh7vfCUqzWDZWnoB+DNhjZruB08ANwI0dy9wPfNrM7iMrx/ynuz/HCNwds24Jf/XGnQ891O39pCHWRgHcemtWZpmZyYK5JkFrrIEB3d1fN7NPA0eBzcBX3f0JMzvUev0rwLeAa4ATwMvAp0ZpzPbt21ldXWVycjKaoD4qd2d1dZXt27eHborUmWaxlDZR3ST6tddeY2VlpTbjv7dv38709DRbt24N3RQRqYl+N4mO6tL/rVu3snv37tDNEBFJUrKX/ouIyHoK6CIiNaGALiJSE8E6Rc3sDNBlAtxcdgFnC2xOCrTOzaB1boZx1nnW3btemRksoI/DzJZ79fLWlda5GbTOzVDWOqvkIiJSEwroIiI1kWpAPxK6AQFonZtB69wMpaxzkjV0ERHZKNUMXUREOiigi4jURNQBPcabU5ctxzrvb63ro2b2fTN7X4h2FmnQOrctt9fM3jCzj1XZvjLkWWcz+7CZPWxmT5jZ/6+6jUXLsW//spn9q5k90lrnkWZtjYWZfdXMnjezx3u8Xnz8cvcoH2RT9f4M+HVgG/AIcGnHMtcA/4/sjkkfAB4K3e4K1vmDwIWt/766Cevcttx3yKZq/ljodlewnS8AfgzMtP7+ldDtrmCd/xL4Yuu/p4BzwLbQbR9jnX8XuAJ4vMfrhcevmDP0xtycus3AdXb377v7C60/f0B2d6iU5dnOAH8K/BPwfJWNK0medb4R+Ka7nwJw99TXO886O7DDspshvIMsoL9ebTOL4+7fJVuHXgqPXzEH9F43nh52mZQMuz5/THaET9nAdTazi4A/BL5SYbvKlGc7vwu40Mz+zcyOm9mBylpXjjzrfCdwCdntKx8DPuPuv6imeUEUHr+img+9Q2E3p05I7vUxs98nC+i/U2qLypdnnb8E3Ozub6R+J6uWPOu8BZgHrgTeDvy7mf3A3X9aduNKkmedPwI8DPxP4DeAb5vZ99z9v0puWyiFx6+YA3oTb06da33M7HLgLuBqd1+tqG1lybPOC8B9rWC+C7jGzF5393+upIXFy7tvn3X3l4CXzOy7wPuAVAN6nnX+FPBXnhWYT5jZU8BvAj+spomVKzx+xVxyefPm1Ga2jezm1Pd3LHM/cKDVW/wBxrg5dSQGrrOZzQDfBG5KOFtrN3Cd3X23u8+5+xzwj8CfJBzMId++/S/Ah8xsi5lNkN18/cmK21mkPOt8iuyMBDP7VeDdwMlKW1mtwuNXtBm6V3hz6ljkXOfbgEng71oZ6+ue8Ex1Ode5VvKss7s/aWYPAI8CvwDucveuw99SkHM73wF83cweIytH3OzuyU6ra2bfAD4M7DKzFeB/A1uhvPilS/9FRGoi5pKLiIgMQQFdRKQmFNBFRGpCAV1EpCYU0EVEakIBXUSkJhTQRURq4r8BdxgQX1jGgSUAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_x0 = [i[0] for _,i in dfData[dfData['label']==0].iterrows()]\n",
    "plot_y0 = [i[1] for _,i in dfData[dfData['label']==0].iterrows()]\n",
    "plot_x1 = [i[0] for _,i in dfData[dfData['label']==1].iterrows()]\n",
    "plot_y1 = [i[1] for _,i in dfData[dfData['label']==1].iterrows()]\n",
    "\n",
    "plt.plot(plot_x0, plot_y0, 'ro', label='x_0')\n",
    "plt.plot(plot_x1, plot_y1, 'bo', label='x_1')\n",
    "plt.legend(loc='best')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来我们将数据转换成 NumPy 的类型，接着转换到 Tensor 为之后的训练做准备"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "np_data = np.array(dfData.values, dtype='float32') # 转换成 numpy array\n",
    "x_data = torch.from_numpy(np_data[:, 0:2]) # 转换成 Tensor, 大小是 [100, 2]\n",
    "y_data = torch.from_numpy(np_data[:, -1]).unsqueeze(1) # 转换成 Tensor，大小是 [100, 1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面我们来实现以下 Sigmoid 的函数，Sigmoid 函数的公式为\n",
    "\n",
    "$$\n",
    "f(x) = \\frac{1}{1 + e^{-x}}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 定义 sigmoid 函数\n",
    "def sigmoid(x):\n",
    "    return 1 / (1 + np.exp(-x))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "画出 Sigmoid 函数，可以看到值越大，经过 Sigmoid 函数之后越靠近 1，值越小，越靠近 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x1ba452cd4c0>]"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAAAsTAAALEwEAmpwYAAAba0lEQVR4nO3dfZQU5Zn38e/lDCAoigoaEFjIEV0w8QUnkBei+BJFjBI1Img0omfRNeQkJs8eSVzzcrImjxqfja4oIhJBXcFIZkQDokRZs0cBhygqgjpiAgMIKAoIwjAz1/PH3cSm6ZnpGaq7umt+n3PqdFfVPdPXVM/8uLm77ipzd0REpPQdEHcBIiISDQW6iEhCKNBFRBJCgS4ikhAKdBGRhCiP64W7d+/u/fr1i+vlRURK0tKlSz9w9x7Z9sUW6P369aO6ujqulxcRKUlm9vem9mnIRUQkIRToIiIJoUAXEUkIBbqISEIo0EVEEqLFQDezaWa20czeaGK/mdldZlZjZq+Z2eDoyxQRkZbk0kN/EBjRzP5zgQGpZTxw7/6XJSIirdXieeju/oKZ9WumyShghofr8C4ys25m1tPd10dVpIgklDvU1cGuXXsvO3d+9ry+Hhoaclsy2zY2htfYs+x5zabWo9qW+TNmGjYMzj472mNJNBOLjgbWpK3XprbtE+hmNp7Qi6dv374RvLSIxKaxETZvhk2b4IMPwrLn+ebNsHUrfPIJbNu277JjRwjturq4f4rCMNt7/cYbizbQLcu2rHfNcPcpwBSAiooK3VlDpJi5w4YNsGIFvPUWrF792bJmDdTWhh5xNl26QNeuey89e8KAAeH5QQdBp05w4IHhMdty4IHQsSN06ABlZS0v5eX7bjvggBCm6Qs0vx7FtphEEei1QJ+09d7Augi+r4gUSn09vPkmLFkSltdfh5Ur4eOPP2tTXg69e0PfvmHIoE+fENLdu0OPHuFxz3LggbH9KO1ZFIE+B5hgZjOBocAWjZ+LFLmGBli6FJ59FhYsCCG+Y0fY160bnHQSjB0LAweG5bjjoFev0OuVotVioJvZo8BwoLuZ1QI/BzoAuPtkYC4wEqgBdgDj8lWsiOyHnTvh6afhD3+AuXM/632ffDJccw0MHQpDhsAxx8Q+dCBtk8tZLmNb2O/A9yKrSESi4w4vvgj33QdVVeEDySOOgAsvhHPOgTPOCMMlkgixXT5XRPJo506YPh0mTQrj4V27wujRYTn99PBBoySOAl0kSXbsCL3x22+H9eth8GC4/34YMwYOPjju6iTPFOgiSdDYCA8/DBMnhiAfPjysn366xsPbEQW6SKlbuhSuvz6cqTJkCMyaBV//etxVSQx0tUWRUlVXBzffHM5OWbMGZsyAl15SmLdj6qGLlKK33w4fcC5bBlddBf/5n+H8cWnXFOgipaaqCq68MkyPnzMHzj8/7oqkSGjIRaRUuMPPfx7OIT/uuDB2rjCXNOqhi5SC+nq49lqYNi0Msdx7r66XIvtQD12k2O3cCRddFML85pvDo8JcslAPXaSY1dXBxRfDvHlh1uf118ddkRQxBbpIsdq9O8zwnDs3zP4cPz7uiqTIachFpBi5w7/8C1RWwl13KcwlJwp0kWL061+Hi2v98pfw/e/HXY2UCAW6SLGZNQv+/d/hiivCh6AiOVKgixSTPTM/hw0LV0nUhbWkFRToIsVi27Ywnf/ww2H27DATVKQVdJaLSDFwDxOHamrg+efhyCPjrkhKkAJdpBg8+CA8+ijccgucemrc1UiJ0pCLSNzWrIEf/jDclGLixLirkRKmQBeJ057zzRsawpT+A/QnKW2nIReROP3+9zB/Ptx9N/TvH3c1UuLUHRCJy8aN8KMfwWmnwb/+a9zVSAIo0EXi8pOfwI4d4TotGmqRCOi3SCQOixaFMfMbbgg3qxCJgAJdpNAaGmDCBOjVK0zxF4mIPhQVKbTp08Pt4x55BLp2jbsaSRD10EUK6dNPw31BhwyBsWPjrkYSRj10kUK65x6orYUZM3ThLYmceugihbJlS7jO+dlnw+mnx12NJJACXaRQbr8dNm+G3/wm7kokoXIKdDMbYWZvmVmNme1zsQkzO9TMnjSzZWa23MzGRV+qSAnbvBnuvBMuuQQGD467GkmoFgPdzMqAScC5wCBgrJkNymj2PeBNdz8RGA7cYWYdI65VpHT913/BJ5/oNEXJq1x66EOAGndf5e51wExgVEYbB7qamQEHA5uB+kgrFSlV27aF3vn558MJJ8RdjSRYLoF+NLAmbb02tS3d3cBAYB3wOvADd2/M/EZmNt7Mqs2setOmTW0sWaTE3HcffPQR3HRT3JVIwuUS6NnOrfKM9XOAV4FewEnA3WZ2yD5f5D7F3SvcvaJHjx6tLFWkBO3cCXfcAWeeCUOHxl2NJFwugV4L9Elb703oiacbB/zRgxrgPeCfoylRpIRNnw7vv6/euRRELoH+MjDAzPqnPugcA8zJaLMaOBPAzI4CjgNWRVmoSMlpbITf/Q5OOSXcjUgkz1qcKeru9WY2AZgPlAHT3H25mV2X2j8Z+BXwoJm9ThiiudHdP8hj3SLF79lnYeVKeOghzQqVgjD3zOHwwqioqPDq6upYXlukIEaOhFdegb//HTrqLF6JhpktdfeKbPs0U1QkH1auhHnz4PrrFeZSMAp0kXy46y7o1AmuvTbuSqQdUaCLRG3LlnB2y2WXwZFHxl2NtCMKdJGo/fd/h3uF6sbPUmAKdJEouYeZoSedBBVZP7cSyRsFukiUqqth2TIYP16nKkrBKdBFojRlCnTpEsbPRQpMgS4Sla1b4dFHYcwYOPTQuKuRdkiBLhKVRx+F7dvDcItIDBToIlG5//5wvfMhQ+KuRNopBbpIFJYvh6VLYdw4fRgqsVGgi0ThoYegrEwfhkqsFOgi+6uhAR5+GEaM0MxQiZUCXWR/LVwIa9fCFVfEXYm0cwp0kf01YwYccghccEHclUg7p0AX2R/bt8Ps2TB6NHTuHHc10s4p0EX2R2VlCHUNt0gRUKCL7I8ZM6BfPxg2LO5KRBToIm22YQP8+c/hVMUD9Kck8dNvoUhbzZ4NjY3h2i0iRUCBLtJWs2bBwIHwhS/EXYkIoEAXaZt16+Avf4FLL9VUfykaCnSRtnj88XB3otGj465E5B8U6CJtMWsWfPGLYchFpEgo0EVaa80aePHFMNwiUkQU6CKt9dhj4VGBLkVGgS7SWo89BoMHwzHHxF2JyF4U6CKt8d57sGSJeudSlBToIq3x+OPh8ZJL4q1DJAsFukhrVFXBySdD//5xVyKyj5wC3cxGmNlbZlZjZhObaDPczF41s+Vm9j/RlilSBN5/H156CS68MO5KRLIqb6mBmZUBk4BvALXAy2Y2x93fTGvTDbgHGOHuq81M9+GS5HnyyTCZ6FvfirsSkaxy6aEPAWrcfZW71wEzgVEZbS4D/ujuqwHcfWO0ZYoUgaoq+Pznde0WKVq5BPrRwJq09drUtnTHAoeZ2UIzW2pmV2b7RmY23syqzax606ZNbatYJA7btsGCBaF3rmu3SJHKJdCz/fZ6xno5cApwHnAOcLOZHbvPF7lPcfcKd6/o0aNHq4sVic28eVBXp+EWKWotjqETeuR90tZ7A+uytPnA3bcD283sBeBE4O1IqhSJW1UV9OgBX/1q3JWINCmXHvrLwAAz629mHYExwJyMNk8AXzezcjPrAgwFVkRbqkhM6urgT3+CCy6AsrK4qxFpUos9dHevN7MJwHygDJjm7svN7LrU/snuvsLMngZeAxqBqe7+Rj4LFymYhQth61YNt0jRy2XIBXefC8zN2DY5Y/124PboShMpElVVcNBBcOaZcVci0izNFBVpTmNjCPQRI6Bz57irEWmWAl2kOS+/DOvXa7hFSoICXaQ5VVVQXg7nnRd3JSItUqCLNKeqCoYPh8MOi7sSkRYp0EWasnJlWDTcIiVCgS7SlCeeCI8XXBBvHSI5UqCLNKWyEioqoE+fltuKFAEFukg269bB4sUabpGSokAXyWZO6uoWupmFlBAFukg2VVUwYAAMHBh3JSI5U6CLZNqyBZ57Ttc+l5KjQBfJNHcu7N6t8XMpOQp0kUxVVXDUUfDlL8ddiUirKNBF0u3aFXroo0bBAfrzkNKi31iRdM89B598ouEWKUkKdJF0VVVw8MFwxhlxVyLSagp0kT0aG8N0/5EjoVOnuKsRaTUFusgeixbBhg0abpGSpUAX2aOqCjp0CD10kRKkQBcBcA8X4zrjDDj00LirEWkTBboIwIoVUFOj4RYpaQp0EQi9c9C1z6WkKdBFIIyfDx0KvXrFXYlImynQRdasgepqXSpXSp4CXaSqKjwq0KXEKdBFKith0CA49ti4KxHZLwp0ad8+/BBeeEFnt0giKNClfXvqKWho0HCLJIICXdq3ykro0wdOOSXuSkT2mwJd2q/t22H+fN1qThJDgS7t1zPPwM6dGj+XxMgp0M1shJm9ZWY1ZjaxmXZfMrMGM/t2dCWK5EllJRx+OJx6atyViESixUA3szJgEnAuMAgYa2aDmmh3KzA/6iJFIrd7Nzz5JJx/PpSXx12NSCRy6aEPAWrcfZW71wEzgVFZ2n0fmA1sjLA+kfx44QX4+GMNt0ii5BLoRwNr0tZrU9v+wcyOBi4EJjf3jcxsvJlVm1n1pk2bWlurSHQqK6FzZzj77LgrEYlMLoGe7eN/z1j/HXCjuzc0943cfYq7V7h7RY8ePXIsUSRijY1huv+IEdClS9zViEQml8HDWqBP2npvYF1GmwpgpoVTv7oDI82s3t2roihSJFKLF8PatXDRRXFXIhKpXAL9ZWCAmfUH1gJjgMvSG7h7/z3PzexB4CmFuRStxx6Djh3DB6IiCdJioLt7vZlNIJy9UgZMc/flZnZdan+z4+YiRaWxER5/PAy36FZzkjA5na/l7nOBuRnbsga5u1+1/2WJ5MnixVBbC7/5TdyViEROM0WlfdFwiySYAl3aDw23SMIp0KX9WLQoDLdccknclYjkhQJd2o8//EHDLZJoCnRpHzTcIu2AAl3aBw23SDugQJf2YdYsDbdI4inQJfnq62HmzBDmGm6RBFOgS/ItWAAbN8Lll8ddiUheKdAl+R55BLp1g5Ej465EJK8U6JJs27eHa5+PHg2dOsVdjUheKdAl2Z54IoS6hlukHVCgS7I9/DD07QvDhsVdiUjeKdAluTZuhGeegcsugwP0qy7Jp99ySa5Zs6ChAb7znbgrESkIBbok1+9/DyefDMcfH3clIgWhQJdkeuWVsFxzTdyViBSMAl2S6YEHwmmKl13WcluRhFCgS/J8+mmYTHTxxXDYYXFXI1IwCnRJnspK+PhjuPrquCsRKSgFuiTPAw9A//5w+ulxVyJSUAp0SZb33oPnnoNx43TuubQ7+o2XZLnvPigrC4Eu0s4o0CU5Pv0Upk6FUaOgd++4qxEpOAW6JMesWfDhhzBhQtyViMRCgS7J4A533w2DBsHw4XFXIxKL8rgLEInEkiWwdCnccw+YxV2NSCzUQ5dkuPtu6NpVF+KSdk2BLqVv7dowfj5uXAh1kXZKgS6l7847obERbrgh7kpEYpVToJvZCDN7y8xqzGxilv2Xm9lrqeVFMzsx+lJFstiyBSZPhksugX794q5GJFYtBrqZlQGTgHOBQcBYMxuU0ew94DR3PwH4FTAl6kJFsrrvPti2Df7t3+KuRCR2ufTQhwA17r7K3euAmcCo9Abu/qK7f5RaXQRoVofk365dYbjlrLNg8OC4qxGJXS6BfjSwJm29NrWtKdcA87LtMLPxZlZtZtWbNm3KvUqRbGbMgHXr1DsXSckl0LOd1OtZG5qdTgj0G7Ptd/cp7l7h7hU9evTIvUqRTHV18B//AUOHwje+EXc1IkUhl4lFtUCftPXewLrMRmZ2AjAVONfdP4ymPJEmTJsGq1fD/fdrIpFISi499JeBAWbW38w6AmOAOekNzKwv8EfgCnd/O/oyRdLs2gW33AJf+5p65yJpWuyhu3u9mU0A5gNlwDR3X25m16X2TwZ+BhwB3GOht1Tv7hX5K1vatalTobYWHnxQvXORNOaedTg87yoqKry6ujqW15YStm0bDBgAxx0HCxcq0KXdMbOlTXWYdXEuKS233gobNsCTTyrMRTJo6r+UjtWr4Y474PLL4UtfirsakaKjQJfS8dOfhsdf/zreOkSKlAJdSsOLL8Ijj8CPfwx9+8ZdjUhRUqBL8du9G669Fvr0gYn7XBtORFL0oagUvzvugDfegDlz4OCD465GpGiphy7FbdUq+OUv4aKL4Pzz465GpKgp0KV4NTTAVVdBhw7hqooi0iwNuUjx+u1v4S9/CTNCe+uKzCItUQ9ditMrr8DNN8PFF8OVV8ZdjUhJUKBL8dm6FcaOhe7dwx2JNCNUJCcacpHi4g5XXw01NbBgARxxRNwViZQMBboUl9/+FmbPDo/Dh8ddjUhJ0ZCLFI8nnwwTh779bfjRj+KuRqTkKNClOCxZApdeGm72rOuci7SJAl3i9/bb8M1vwuc+B089BQcdFHdFIiVJgS7xevvtz8bKn34ajjoq1nJESpkCXeKzJ8zr6+H55+HYY+OuSKSkKdAlHkuWwLBhn4X58cfHXZFIyVOgS+FVVYWeedeuYWq/wlwkEgp0KZyGhs+unHjCCfDSS+FmzyISCU0sksLYuDHcC3TBArjiCpg8Gbp0ibsqkURRD13yyx0eewy+8AX43/+FqVNh+nSFuUgeKNAlf1avDldLvPTScB/QJUvgmms0aUgkTxToEr2tW+GnPw3j4/PmwW23waJF8MUvxl2ZSKJpDF2i89FHMGlSuLvQBx/Ad74Dt9wSeucikncKdNl/K1fClClw//3wySdw3nnwi19ARUXclYm0Kwp0aZuPPoInnoAHHggfdpaXw+jRcOON4ZREESk4BbrkbvVq+NOfoLIyzO6srw/T9W+7LdwmTtdhEYmVAl2ya2yEd9+FxYtDeC9cCKtWhX3HHgs//jFceCEMGaKzVkSKhAK9vXOHDRvCLd/eeQeWLQs3aH7lFdi2LbQ57DA47TT4wQ/grLNg4ECFuEgRyinQzWwEcCdQBkx19/+bsd9S+0cCO4Cr3P2vEdcqrdXQAB9/HAJ73TpYvz48rlsHa9eGHndNzWfBDdC5M5x0UpjNOXgwnHJKGBM/QGe4ihS7FgPdzMqAScA3gFrgZTOb4+5vpjU7FxiQWoYC96YeBUIvuL5+72X37qbX9zzfvRs+/RR27AjLnueZj9u3h+D+6KO9l/SgTnfIIdCrF/TrF654OGBAWI45Bvr3h7KyQh4dEYlILj30IUCNu68CMLOZwCggPdBHATPc3YFFZtbNzHq6+/rIK376abjhhvDcfe8l27amthfq6xsawpIPnTqFKfRdukC3bmFopG9fOPHE8HzPtqOOCgHeqxf07Kk7AokkVC6BfjSwJm29ln1739naHA3sFehmNh4YD9C3rZNNDj00zDjcM4ZrtveSbVtr2kb99eXley8dOmR/3tR6ly5hGCTzsXNnDYOIyF5yCfRsn355G9rg7lOAKQAVFRX77M/JV74SFhER2UsuXbxaoE/aem9gXRvaiIhIHuUS6C8DA8ysv5l1BMYAczLazAGutODLwJa8jJ+LiEiTWhxycfd6M5sAzCectjjN3Zeb2XWp/ZOBuYRTFmsIpy2Oy1/JIiKSTU7nobv7XEJop2+bnPbcge9FW5qIiLSGTpMQEUkIBbqISEIo0EVEEkKBLiKSEObetvk9+/3CZpuAv7fxy7sDH0RYTlSKtS4o3tpUV+uortZJYl3/5O49su2ILdD3h5lVu3vR3d+sWOuC4q1NdbWO6mqd9laXhlxERBJCgS4ikhClGuhT4i6gCcVaFxRvbaqrdVRX67SrukpyDF1ERPZVqj10ERHJoEAXEUmIog10M7vEzJabWaOZVWTs+4mZ1ZjZW2Z2ThNff7iZPWtm76QeD8tDjbPM7NXU8jcze7WJdn8zs9dT7aqjriPL6/3CzNam1TayiXYjUsewxswmFqCu281spZm9ZmaVZtatiXYFOV4t/fypy0Hfldr/mpkNzlctaa/Zx8yeN7MVqd//H2RpM9zMtqS9vz/Ld11pr93sexPTMTsu7Vi8amZbzeyHGW0KcszMbJqZbTSzN9K25ZRFkfw9untRLsBA4DhgIVCRtn0QsAzoBPQH3gXKsnz9bcDE1POJwK15rvcO4GdN7Psb0L2Ax+4XwP9poU1Z6th9HuiYOqaD8lzX2UB56vmtTb0nhTheufz8hEtCzyPckevLwOICvHc9gcGp512Bt7PUNRx4qlC/T615b+I4Zlne1/cJk28KfsyAU4HBwBtp21rMoqj+Hou2h+7uK9z9rSy7RgEz3X2Xu79HuAb7kCbaTU89nw58Ky+FEnolwGjg0Xy9Rh784+bf7l4H7Ln5d964+zPuXp9aXUS4s1Vccvn5/3Hzc3dfBHQzs575LMrd17v7X1PPtwErCPfnLRUFP2YZzgTedfe2zkLfL+7+ArA5Y3MuWRTJ32PRBnozmrohdaajPHXXpNTjkXms6evABnd/p4n9DjxjZktTN8ouhAmp//JOa+K/eLkex3y5mtCTy6YQxyuXnz/WY2Rm/YCTgcVZdn/FzJaZ2TwzO75QNdHyexP379UYmu5YxXXMcsmiSI5bTje4yBczWwB8Lsuum9z9iaa+LMu2vJ17mWONY2m+d/41d19nZkcCz5rZytS/5HmpC7gX+BXhuPyKMBx0dea3yPK1+30cczleZnYTUA880sS3ifx4ZSs1y7Y23fw8H8zsYGA28EN335qx+6+EIYVPUp+PVAEDClEXLb83cR6zjsAFwE+y7I7zmOUikuMWa6C7+1lt+LJcb0i9wcx6uvv61H/5NuajRjMrBy4CTmnme6xLPW40s0rCf6/2K6ByPXZmdj/wVJZdebmxdw7H67vAN4EzPTV4mOV7RH68sijam5+bWQdCmD/i7n/M3J8e8O4+18zuMbPu7p73i1Dl8N7EecP4c4G/uvuGzB1xHjNyy6JIjlspDrnMAcaYWScz60/4V3ZJE+2+m3r+XaCpHv/+OgtY6e612Xaa2UFm1nXPc8IHg29kaxuVjDHLC5t4vVxu/h11XSOAG4EL3H1HE20KdbyK8ubnqc9jHgBWuPv/a6LN51LtMLMhhL/jD/NZV+q1cnlv4rxhfJP/U47rmKXkkkXR/D3m+1Pfti6EIKoFdgEbgPlp+24ifCL8FnBu2vappM6IAY4A/gy8k3o8PE91Pghcl7GtFzA39fzzhE+slwHLCUMP+T52DwGvA6+lfil6ZtaVWh9JOIvi3QLVVUMYJ3w1tUyO83hl+/mB6/a8n4T/Bk9K7X+dtLOt8ljTMMJ/tV9LO04jM+qakDo2ywgfLn8133U1997EfcxSr9uFENCHpm0r+DEj/IOyHtidyq9rmsqifPw9auq/iEhClOKQi4iIZKFAFxFJCAW6iEhCKNBFRBJCgS4ikhAKdBGRhFCgi4gkxP8Hkjp4rdpEY8MAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 画出 sigmoid 的图像\n",
    "\n",
    "plot_x = np.arange(-10, 10.01, 0.01)\n",
    "plot_y = sigmoid(plot_x)\n",
    "\n",
    "plt.plot(plot_x, plot_y, 'r')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "x_data = Variable(x_data)\n",
    "y_data = Variable(y_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在 PyTorch 当中，不需要我们自己写 Sigmoid 的函数，PyTorch 已经用底层的 C++ 语言为我们写好了一些常用的函数，不仅方便我们使用，同时速度上比我们自己实现的更快，稳定性更好\n",
    "\n",
    "通过导入 `torch.nn.functional` 来使用，下面就是使用方法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from torch import sigmoid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 定义 logistic 回归模型\n",
    "w = Variable(torch.randn(2, 1), requires_grad=True) \n",
    "b = Variable(torch.zeros(1), requires_grad=True)\n",
    "\n",
    "def logistic_regression(x):\n",
    "    # y = sigmoid(x*w+b)\n",
    "    return sigmoid(torch.mm(x, w) + b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在更新之前，我们可以画出分类的效果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x1ba45291460>"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAAAsTAAALEwEAmpwYAAAxzUlEQVR4nO3deXwU5f3A8c9DAJFDxYCiRghW9MchoAQUBAUpigioVVtpxMoVAbFqrRaFWlTwrgciQgSKFLzxQOQQSpVDQUJFuUQROQLKkch9Jvn+/tgNLnGP2ezszszu9/16zSvZmdmd78zOfufZZ559HiMiKKWUSl4VnA5AKaVUfGmiV0qpJKeJXimlkpwmeqWUSnKa6JVSKslVdDqAYGrVqiWZmZlOh6GUUp6xbNmynSJSO9gyVyb6zMxM8vLynA5DKaU8wxizMdQyrbpRSqkkp4leKaWSnCZ6pZRKcq6sow/m6NGj5Ofnc+jQIadDSXpVqlQhIyODSpUqOR2KUsoGERO9MeZsYBJQBygBckXkhTLrGOAFoAtwALhNRP7nX9bZvywNGCciT5Qn0Pz8fGrUqEFmZia+zal4EBEKCgrIz8+nfv36ToejlLKBlaqbIuBeEWkIXALcYYxpVGadq4EG/ikHeBnAGJMGvORf3gjoEeS5lhw6dIj09HRN8nFmjCE9PV2/OSmVRCImehH5sbR0LiJ7gTXAWWVWuxaYJD6LgVOMMWcArYB1IrJeRI4Ab/jXLRdN8omhx1mp5BLVzVhjTCZwIbCkzKKzgM0Bj/P980LND/baOcaYPGNM3o4dO6IJSymlPGneD/N4YfELkVeMkeVEb4ypDkwF7haRPWUXB3mKhJn/65kiuSKSJSJZtWsH/XGXp2zYsIHXXnvt2OPly5czY8aMY4+nTZvGE0+U63bFr9x222288847APTt25fVq1fb8rpKqfjYfWg3t394Ox0ndWTssrEcLjoc1+1ZSvTGmEr4kvwUEXk3yCr5wNkBjzOArWHmJ71Iib579+4MHjzY9u2OGzeORo3KdRtEKZUAM76bQZOXmzDuy3H8tfVfycvJ44SKJ8R1mxETvb9FzXhgjYg8G2K1acCtxucSYLeI/AgsBRoYY+obYyoDN/vX9aRJkybRtGlTmjVrRs+ePYHjS9MA1atXB2Dw4MEsWLCA5s2b8+STT/LQQw/x5ptv0rx5c958800mTpzIoEGDjr3Gn//8Z9q0acM555xz7PVKSkoYOHAgjRs3pmvXrnTp0uW4bQXTvn37Y91HVK9enSFDhtCsWTMuueQStm3bBsCOHTu44YYbaNmyJS1btmTRokX2Hiil1K8UHizkT+//iWteu4aTTziZz/t8ztNXPk3VSlXjvm0r7egvBXoCK4wxy/3zHgTqAojIGGAGvqaV6/A1r+zlX1ZkjBkEzMbXvHKCiKyKNei7Z93N8p+WR1wvGs3rNOf5zs+HXL5q1SpGjBjBokWLqFWrFoWFhWFf74knnuCZZ55h+vTpAJx++unk5eUxatQoACZOnHjc+j/++CMLFy7km2++oXv37tx44428++67bNiwgRUrVrB9+3YaNmxI7969Le/T/v37ueSSSxgxYgT3338/r7zyCkOHDuWuu+7innvuoW3btmzatImrrrqKNWvWWH5dpVR03lvzHgM+GkDBwQKGthvK0MuGxr0UHyhioheRhQSvaw9cR4A7Qiybge9C4Gnz5s3jxhtvpFatWgCceuqptr7+ddddR4UKFWjUqNGxkvfChQu56aabqFChAnXq1KFDhw5RvWblypXp2rUrAC1atGDOnDkAzJ0797h6/D179rB3715q1Khh094opQC279/OoBmDeHv121xY50Jm3TKL5nWaJzwOz/wyNlC4kne8iEjQZocVK1akpKTk2DpHjhwp1+ufcMIvV/fSAdtjHbi9UqVKx2JOS0ujqKgI8FUJff7555x44okxvb5SKjgR4fWVr/PnmX9m75G9jLhiBPe1uY9Kac782lz7urGoY8eOvPXWWxQUFAAcq7rJzMxk2bJlAHzwwQccPXoUgBo1arB3795jzy/72Iq2bdsydepUSkpK2LZtG5988okNewJXXnnlsSok8N0oVkrZY8ueLVz7xrVkv5vNuaeey5e3f8mD7R50LMmDJnrLGjduzJAhQ7j88stp1qwZf/nLXwDo168fn376Ka1atWLJkiVUq1YNgKZNm1KxYkWaNWvGc889R4cOHVi9evWxm7FW3HDDDWRkZNCkSRNuv/12Lr74Yk4++eSY92XkyJHk5eXRtGlTGjVqxJgxY2J+TaVSnYgw4csJNB7dmDnr5/DPK//Jot6LaFTb+VZwJtbqgXjIysqSsgOPrFmzhoYNGzoUkXP27dtH9erVKSgooFWrVixatIg6derEfbuperyVKo+NuzbS78N+zFk/h3Z12zG++3gapDdIaAzGmGUikhVsmSfr6FNJ165d2bVrF0eOHOHvf/97QpK8UsqaEilhTN4Y/jb3b4gIL3V5if5Z/alg3FVZoone5eyql1dK2Wtd4Tr6TOvD/I3z6XROJ3K75ZJ5SqbTYQWliV4ppaJQXFLMC0teYOi8oVROq8z47uPp1byXqzsD1ESvlFIWrd6xmt4f9GbJliV0O68bY7qO4cwaZzodVkSa6JVSKoKjxUd5+rOnefjTh6lRuQZTfjeFHk16uLoUH0gTvVJKhbH8p+X0/qA3X/70Jb9v/HtevPpFTqt2mtNhRUUTvVJKBXG46DAjFozg8YWPk35iOlN/P5XfNfyd02GVi7vaANlpyhTIzIQKFXx/p0xJ2KZfffVVGjRoQIMGDXj11VcTtl2llD2W5C/hotyLeHT+o/Ro0oPVd6z2bJKHZC3RT5kCOTlw4IDv8caNvscA2dlx3XRhYSEPP/wweXl5GGNo0aIF3bt3p2bNmnHdrlIqdgePHuSh/z7Es4uf5cwaZ/LRHz+iS4MuTocVs+Qs0Q8Z8kuSL3XggG9+OS1dupSmTZty6NAh9u/fT+PGjVm5cuWv1ps9ezadOnXi1FNPpWbNmnTq1IlZs2aVe7tKqcRYsHEBzcY045nPn6HPhX1YOWBlUiR5SNYS/aZN0c23oGXLlnTv3p2hQ4dy8OBBbrnlFpo0afKr9bZs2cLZZ/8yqFZGRgZbtmwp93aVUvG178g+Hpj7AKOWjqL+KfWZ23MuHc/p6HRYtkrORF+3rq+6Jtj8GDz00EO0bNmSKlWqMHLkyKDrBOs7yCtNsJRKNXPXz6Xfh/3YuGsjd7a6k8c6Pkb1ytWdDst2VoYSnGCM2W6M+XU9hW/5fcaY5f5ppTGm2Bhzqn/ZBmPMCv+yvGDPj4sRI6BqmeG5qlb1zY9BYWEh+/btY+/evRw6dCjoOhkZGWzevPnY4/z8fM480/0/qFAqlew+tJt+0/rR6d+dqJxWmfm95jPy6pFJmeQBXwk03ARcBlwErLSwbjdgXsDjDUCtSM8rO7Vo0ULKWr169a/mhTV5ski9eiLG+P5Onhzd84Po1q2bTJkyRYYPHy533HFH0HUKCgokMzNTCgsLpbCwUDIzM6WgoCDmbSda1MdbKY+Yvna6nPXPs6TCwxXkvo/vkwNHDjgdki2APAmRU60MJTjfGJNp8brRA3g96qtNPGRn29rCZtKkSVSsWJE//vGPFBcX06ZNG+bNm8cVV1xx3Hqnnnoqf//732nZsiXgq+6xe9hBpVT0Cg4UcPfsu5n89WSanNaE9/7wHi3Paul0WAlhqT96f6KfLiK/vvv4yzpVgXzgXBEp9M/7AfgZEGCsiOSGeX4OkANQt27dFhvL1LFr/+iJpcdbJZOpq6cycMZACg8W8mDbBxly2RAqp1V2OixbJao/+m7AotIk73epiGw1xpwGzDHGfCMi84M92X8RyAXfwCM2xqWUSlHb9m1j0MxBvLP6HS464yI+vuVjmtVp5nRYCWdnor+ZMtU2IrLV/3e7MeY9oBUQNNF7zYoVK+jZs+dx80444QSWLFniUERKqVIiwpQVU7hr1l3sO7KPxzs+zl/b/JWKFZKzoWEktuy1MeZk4HLgloB51YAKIrLX//+VwCN2bM8NLrjgAh1UWykX2rJnC/0/6s/0b6fTOqM147uPp2Ht1K6GjJjojTGvA+2BWsaYfOAfQCUAESkdVfp64GMR2R/w1NOB9/xtyCsCr4mI/kRUKRUXIsL4L8dz78f3crT4KM9d9Rx3trqTtAppTofmOCutbnpYWGciMLHMvPVA6lWGKaUS7oeffyBneg5z18+lfWZ7xnUbx29O/Y3TYblGalZYKaWSQomUMHrpaAbPHYwxhpeveZmcFjmuG5zbaZrolVKe9G3Bt/Sd1pcFmxZw1W+uIrdbLnVPjq2bk2SVtJc9B7ujp3Pnzpxyyil07do1cRtVKkUUlRTx9KKnaTamGSu2r+Bf1/6LmdkzNcmHkZQlege7owfgvvvu48CBA4wdOzb+G1MqhazavopeH/Ri6dalXHv+tbx8zcucUeMMp8NyvaQs0cehO3rL/dEDdOzYkRo1apR/Y0qp4xwtPsrw+cO5cOyFrP95Pa/f8Drv/eE9TfIWJWWJPg7d0Vvuj14pZa8vf/yS3tN6s/yn5fyh8R948eoXqV2tttNheUpSJvo4dUdvqT96pZQ9Dhcd5tH5j/LEwieoXa027/7+Xa5veL3TYXlSUlbdxKk7ekv90SulYrckfwkXjr2QEQtG0LNZT1YPXK1JPgZJmeizsyE3F+rVA2N8f3NzY78Rm5OTw6OPPkp2djZ/+9vf7AlWKXXMgaMHuHf2vbSZ0IZ9R/YxM3sm/7r2X9Q8sabToXlaUlbdgO3d0Vvujx6gXbt2fPPNN+zbt4+MjAzGjx/PVVddZV8wSiWh+Rvn02daH9YVrqN/i/482elJTjrhJKfDSgpJm+jtduutt3LrrbcCkJaWFraXygULFiQqLKU8b+/hvQyeO5jReaM5p+Y5zLt1Hh3qd3A6rKSiiV4p5Zg538+h74d92bx7M3dffDfDrxhOtcrVnA4r6WiiLyftj16p8tt1aBf3zr6XCcsncH76+SzsvZA2Z7dxOqyk5alELyL4uz12XDL3R29leEmlyuvDtR9y+/Tb2b5/O4MvHcw/2v+DKhWrOB1WUvNMoq9SpQoFBQWkp6e7JtknIxGhoKCAKlX0g6fstfPATu6adRevrXiNC067gA97fEiLM1s4HVZK8Eyiz8jIID8/nx07djgdStKrUqUKGRkZToehkoSI8M7qd7hjxh3sOrSLh9s/zOC2g5NucG43szLC1ASgK7BdRH71m39jTHvgA+AH/6x3ReQR/7LOwAtAGjBORJ4ob6CVKlWifv365X26UsoBP+37iTtm3MG7a94l68ws/tP9P1xw+gVOh5VyrJToJwKjgElh1lkgIsf1yWuMSQNeAjoB+cBSY8w0EVldzliVUh4hIkz+ejJ3zbqLA0cP8ORvn+Qvrf+SsoNzO83KUILzjTGZ5XjtVsA6/5CCGGPeAK4FNNErlcTy9+Rz+/TbmfHdDNqc3YYJ3Sdwfq3znQ4rpdnVBUJrY8xXxpiZxpjG/nlnAZsD1sn3zwvKGJNjjMkzxuRpPbxS3iMivLLsFRqPbswnGz7hhc4vMP+2+ZrkXcCO71H/A+qJyD5jTBfgfaABEKxpTMh2eyKSC+QCZGVlafs+pTxk/c/r6fdhP+b9MI8OmR0Y130c59Q8x+mwlF/MJXoR2SMi+/z/zwAqGWNq4SvBnx2wagawNdbtKaXco0RKeHHJi1zw8gUs3bKUsV3H8p9b/6NJ3mViLtEbY+oA20REjDGt8F08CoBdQANjTH1gC3Az8MdYt6eUcoe1O9fSZ1ofFm1eROdzO5PbNZezTz478hNVwllpXvk60B6oZYzJB/4BVAIQkTHAjcAAY0wRcBC4WXw/rSwyxgwCZuNrXjlBRFbFZS+UUglTVFLEs58/y0P/fYiqlary6nWv0rNpT/0ho4sZN/7cPSsrS/Ly8pwOQylVxoptK+g9rTd5W/O47v+uY3SX0Tpuq0sYY5aJSFawZdqoVSkV0ZHiIzyx8AmGzx/OKVVO4c0b3+SmRjdpKd4jNNErpcJatnUZvaf15uttX9OjSQ9e6PyCDs7tMZrolVJBHSo6xCOfPsJTi57itGqn8cHNH9D9/O5Oh6XKQRO9UupXPt/8Ob2n9eabnd/Qq3kvnr3qWU6pcorTYaly0kSvlDrmwNEDDJ03lOcXP8/ZJ5/N7Ftmc+VvrnQ6LBUjTfRKKQA+2fAJfaf15fufv2dA1gCe/O2T1DihhtNhKRtoolcqxe05vIfBcwfzct7L/Kbmb/jkT59weeblToelbKSJXqkUNmvdLHI+zCF/Tz73XHIPw68YTtVKVZ0OS9lME71SKejngz/zl4//wsTlE2lYqyGLei+i9dmtnQ5LxYkmeqVSzAfffED/j/qzY/8OHmj7AA9d/pAOzp3kNNErlSJ27N/Bn2f9mTdWvkHT05vy0R8/4qIzLnI6LJUAmuiVSnIiwlur3mLQzEHsPrRbB+dOQZrolUpiP+79kYEzBvL+N+/T8syWTLh2Ak1Oa+J0WCrBNNErlYREhElfTeLu2XdzqOgQT/32Ke5pfY8Ozp2i9F1XKsls2r2J26ffzqx1s2hbty3ju4/nvPTznA5LOUgTvVJJokRKyF2Wy31z7kNEePHqFxnYciAVTMwjhiqPszLC1ASgK7BdRH5VuWeMyQb+5n+4DxggIl/5l20A9gLFQFGoTvGVUrH5vvB7+n3Yj/9u+C+/Pee3vNLtFTJPyXQ6LOUSVkr0E4FRwKQQy38ALheRn40xVwO5wMUByzuIyM6YolRKBVVcUsyoL0bx4LwHqVihIq90e4U+F/bRAUHUcSImehGZb4zJDLP8s4CHi4EMG+JSSkXwzc5v6DOtD59t/owuDbowtutYMk7Sj5/6Nbsr7/oAMwMeC/CxMWaZMSYn3BONMTnGmDxjTN6OHTtsDkup5FFUUsSTC5+k+ZjmrNmxhknXTWJ6j+ma5FVItt2MNcZ0wJfo2wbMvlREthpjTgPmGGO+EZH5wZ4vIrn4qn3Iyspy34jlSrnAim0r6PVBL5b9uIzfNfwdL3V5iTrV6zgdlnI5W0r0xpimwDjgWhEpKJ0vIlv9f7cD7wGt7NieUqnmSPERhn0yjBa5Ldi8ZzNv3/Q2U38/VZO8siTmEr0xpi7wLtBTRL4NmF8NqCAie/3/Xwk8Euv2lEo1eVvz6P1Bb1ZsX0H2Bdk83/l5alWt5XRYykOsNK98HWgP1DLG5AP/ACoBiMgY4CEgHRjtv9Nf2ozydOA9/7yKwGsiMisO+6BUUjpUdIhhnwzj6c+epk71Oky7eRrdzu/mdFjKg6y0uukRYXlfoG+Q+euBZuUPTanU9dnmz+j9QW/WFqylz4V9eObKZ3RwblVu+stYpVxk/5H9DJk3hJFLRlL35Lp8fMvHdPpNJ6fDUh6niV4pl5j3wzz6TuvLD7t+YFDLQTz+28epXrm602GpJKCJXimH7T60m/vn3E/u/3I599RzmX/bfNrVa+d0WCqJaKJXykEzv5tJzvQctu7dyl9b/5WHOzysg3Mr22miV8oBhQcLuWf2PUz6ahKNajfinZve4eKMiyM/Ualy0ESvVIK9t+Y9Bnw0gJ0HdjK03VCGXjaUEyqe4HRYKolpolcqQbbv386dM+/krVVv0bxOc2Zmz+TCMy50OiyVAjTRKxVnIsKbq97kzpl3sufwHoZ3GM79l95PpbRKToemUoQmeqXiaOverQz4aADT1k6j1VmtmNB9Ao1Pa+x0WCrFaKJXKg5EhInLJ3LP7Hs4XHyYZzo9w92X3E1ahTSnQ1MpSBO9UjbbuGsjOdNz+Pj7j2lXtx3ju4+nQXoDp8NSKUwTvVI2KZESxuaN5f659yMivNTlJfpn9dfBuZXjNNErZYN1hevoO60vn278lE7ndCK3W64Ozq1cQxO9UjEoLilm5JKRDJk3hMpplRnffTy9mvfSwbmVq2iiV6qcVu9YTZ9pfVicv5iu53VlzDVjOOuks5wOS6lfiVh5aIyZYIzZboxZGWK5McaMNMasM8Z8bYy5KGBZZ2PMWv+ywXYGrpRTjhYf5bEFj3Hh2Av5ruA7Xvvda0y7eZomeeVaVu4STQQ6h1l+NdDAP+UALwMYY9KAl/zLGwE9jDGNYgk27qZMgcxMqFDB93fKFKcjUi6z/KflXDzuYobMG0L387uzauAqelzQQ6tqlKtFTPQiMh8oDLPKtcAk8VkMnGKMOQPfQODrRGS9iBwB3vCv605TpkBODmzcCCK+vzk5muwVAIeLDvPQfx+i5Sst2bp3K1N/P5W3b3qb06ufHvVraXlCJZod7b7OAjYHPM73zws1352GDIEDB46fd+CAb36c6AfeG77Y8gUtclvw6PxH6dGkB6vvWM3vGv6uXK+l5QnlBDsSfbDvrBJmfvAXMSbHGJNnjMnbsWOHDWFFadOm6ObHSD/wiVP2gjpwoLUL7MGjB7l/zv20Ht+aXYd28dEfP2LS9ZM49cRTyx1LecsTiS4UaCEkyYhIxAnIBFaGWDYW6BHweC1wBtAamB0w/wHgASvba9GihcTV5Mki9eqJGOP7W/rYl3OPn+rVi0sICd5cypo8WaRq1eDHunSqWtW3XqAFGxdIg5ENhGFI3w/6yq6Du2yJx5jgMRgT3T4Ei9kuid6esgeQJ6FyeKgFx60UPtFfA8zEV4K/BPjCP78isB6oD1QGvgIaW9leXBN9qLN4wICEnt3l+cCr6IW6oIa6wO49vFfunHGnmGFGMp/PlDnfz0lIPOEu8IkuFGghxJtiSvTA68CPwFF89ex9gP5Af/9yg691zffACiAr4LldgG/9y4ZE2lbpFNdEH+4sDlbSdyAMZZ9QF9RgF9i538+VzOczxQwzMuijQbL38F7b4ylPaTnRhQIthHhTzCX6RE9xTfQuOYv163FiWC3RV6+9UxiGnPfiebJg44K4xhRteUJL9MqKcIk+9Xpbqls3uvlxkp0NublQrx4Y4/ubm+ubr+wzYgRUjTDWtql0gH3t/sx9be5j+e3LaVu3bVxjys6GDRugpMT3t/Q9D3UDNNg+VK3qmx8Pid6eSoBQVwAnJ0fq6LUonbTKlqAHDBDJOLtIMCXCyT/IWX+6T77I/8LxGMOdlgmsVYy4vUTHoqxBq27K0DM1pb2z6h057enTpOIjFeXv8/4uh44ecjokz1SXaDnJvcIleuNb7i5ZWVmSl5fndBgqyWzbt41BMwfxzup3uOiMi5jQfQLN6jRzOizAV10T7KNojK+Kxy0yM32/+SirXj1fNZRyjjFmmYhkBVuWenX0KuWICFO+nkKj0Y2YtnYaj13xGEv6LnFNkof43Tqy+4dPwZJ8uPnKHTTRq6S2Zc8Wur/RnVveu4Xz0s9j+e3LeaDdA1Ss4K4euu28AVqa3I2Bnj3t/fV1Woghb0PNV+6giV4lRKJ/Ui8ijPvfOBqNbsR/1v+HZ698loW9FtKwdsP4bric7GqFFdi1Bvy6OijW7puKi6Ob71Yp18VDqMp7J6e434xNcU604EjkDbwffv5Bfjvpt8Iw5PJ/XS7fFXwXnw25kJXfDcTykxGv3DQOJ1lvKKOtblQpJ07yRCWH4pJiGbVklFQbUU2qP1ZdXvriJSkuKbZ3Iy5n9ZfA5b3AezFJli3YpKd7/2IVjCZ6j4pHyduJElkifoz87c5vpd2EdsIw5Mp/Xykbft5g34t7iNVfAseSoL3UOtlKp3bx/nF8oo6XJnoPilfJyYkeIOJ5cSkqLpJnFj0jVYZXkZMfP1km/G+CTJ5c4plEZLdg5024Ur7XS7GRRHPhi8exSOQ3IE30HhSv5OhEiT5eJ/vKbSvl4lcuFoYh3V/vLlv2bHFl1YKTv2pNTw9dVRHvC7wbWK3Kitc5ksjPmyZ6D4pXydupRGhnsjtSdEQe/fRRqfxoZUl/Ml1eX/G6lJSUiIj7bhY60Zd8YJKvVCl8gkvVEn16emIuvon8Bq2J3oPimbC8VMda1v+2/k+aj2kuDEP+8PYfZNu+bcctd0nnpMck8sITTX20G77pJILT3/C0RK+JPiynT1C3OXT0kAz5zxBJezhN6jxTR95d/W7Q9dxWok/khSfa+uhUOZecLNhoHb0m+oi8XPK20+LNi6XRS42EYcit790qBQcKQq7rtgtkIi880TStVInjmVY3QGd8Y8GuAwYHWX4fsNw/rQSKgVP9yzbgG3lqebhAAidN9EpEZP+R/XLv7HulwsMVJOPZDJnx7QxLz3PTBTKRFx4rJfpU/laY7GJK9EAavqEAz+GXsV8bhVm/GzAv4PEGoFak7QROmujVpxs+lXNHnisMQ/p/2F92H9rtdEjllqgLT7CLSuXKvhuPbrjoqfgKl+it9HXTClgnIutF5AjwBnBtmPV74BtnVqmo7T28l0EzBnH5xMspLilm3q3zeLnry5x0wkkJj8Wu/lBCjShlt2D95UyYADt3xn/byt2sdOF3FrA54HE+cHGwFY0xVfFV8wwKmC3Ax8YYAcaKSG6I5+YAOQB1Ezysn3KHOd/Pod+H/di0exN3X3w3w68YTrXK1RyJpbRzsAMHfI9Le34EdyfL7Gx3x6ecYaVEb4LMkxDrdgMWiUhhwLxLReQi4GrgDmPMZcGeKCK5IpIlIlm1a9e2EJZKFrsO7aLvtL5cOflKqlSswsLeC3mu83OOJXnw9fBYmuRLxdrzo1JOsZLo84GzAx5nAFtDrHszZaptRGSr/+924D18VUEqkhTpR3X6t9NpPLoxE5dPZPClg1nefzltzm7jdFhs2hTd/HhJkdNAxVuoynv55WZqRWA9UJ9fbsY2DrLeyUAhUC1gXjWgRsD/nwGdI23TFTdjU6XxrUN27t8p2VOzhWFIk9FNZOmWpU6HdBw39HCYAqeBshE2NK/sAnyLr/XNEP+8/kD/gHVuA94o87xz/BeGr4BVpc+NNDme6J3+hLntVz82e3vV28cG5x7232FyuOiw0yEdZ/JkX2uVsoe/UqXEJtkkPw08w03NdcOJOdEnenI80Tv9CXPb7/ht8tPen+SGN28QhiEtxraQr376yumQggrXP0oiefk08EpyjMTpMl80wiV6HUowmHhV0FqtcI3XSNExKm99sYgw+evJNBrdiOnfTueJjk+wuO9imp7e1JF4Ign1NhcWBp8fLy49DSIKHM5QxJ6xap2SNDflQ10BnJwcLdFPniySlmZ/iT6aooELixHlDWnz7s1yzZRrhGFI63GtZc2ONY7GY4XTX+hKOX0alLdU7pbjZwcvfatCq24sCtf9X6yfsGjPfpd99402/JKSEsnNy5WTHj9Jqo6oKi8sfkGKiosciycaTifYsrE4cRrEcgy8lBwj8dJFSxO9VaHe1bQ0bw7tZKNowl9fuF46vtpRGIa0n9he1hWsczSe8nDZdTbhwn0UIh0TLyXHSNx00Y9EE71VkbJHLJ9+j5/9VsIvLimWkYtHStURVaXGYzVkzNIxcRucO1I8qZ6oY2WlJ0wP1TzGxCvnkiZ6q8Jlj1jPXo+f/ZHCX7tzrVw6/lJhGNJ5cmfZuGujY/Fo516xs9q3vUdqHl3PjuOlid6qcNnDjhK5x8/+YOEfLT4qTy18SqoMryKnPHGKvLr81WPD+jkRj4h212sHq6NVeaTm0dXsKgNqoo9GqOxhd6Wwx5O+iMiKbSukZW5LYRhy3RvXydY9W50OSUR0AA67BJ6i8WiIpnzsqtXVRG8HO+vYPV6Nc6ToiDzyySNS6ZFKUvup2vLmyjcTVoq3wmq1Q6qXRqMpa3j8lHU1u8qQmujtYOeZ7uEbs8u2LpNmLzcThiE93ukh2/dtFxF3fUGxWu1gpQWJl8Q7cbvpPQ7G7fGFoiV6t7HrTPJgU8uDRw/Kg3MflLSH0+SMZ86Q99e8f2yZG0t7gW9Verqvn5pkrrOP9j2IpfmkG7nxHAyMLVTamDw5eAd6WkefDDxWov988+fScFRDYRhy2/u3SeGBwuOWe2F3kr2+Odr3IJbmk27k1nMw2tZh4Ev8dre6Mb7l7pKVlSV5eXlOhxE/ZYcvAqha1TcOnIuGBzpw9ABD5w3l+cXPk3FSBq90e4Wrzr3qV+tVqOA7RcsyxjeEndt4LV4rot2nzExfHzSR1KvnG4LQ7dz6noY6zvXq+f6GWlaeY26MWSYiWcGWaadmTgg2uKfLkvwnGz6h6ctNeW7xc/TP6s/KgSuDJnnwXudbXovXimj3acQIX9kikkQPtFJebn1Pw/WPmMjBbTTRO8XuEaNt6spx7+G9DPxoIB1e7YAg/PdP/2X0NaPDDs4dLGlUreqb70bxjNepEaGi3aeyZY20tODrOZ0orXLrORjuApTQi1OoOh0np6Svo7ebTXeiZn03S+o+V1fMMCP3zLpH9h3eF1UIXmrxEI94nb4hGMs+OR27HcregLf7l9DlOb7R1tHHcsyxYYSpzsBaYB0wOMjy9sBuYLl/esjqc4NNSZXoE5EBY7wTVXigUHq930sYhvzfqP+TzzZ9Zn+MIXjtAhGOW28IWpUs70U8LlqxvGakVjd2HfOYEj2Qhm8IwXP4ZczYRmXWaQ9ML89zg01Jk+gTVUyKobnm+2velzOeOUPSHk6TB+c+KAePHrQ3tjCSoRQZyIOtZpNSPC64XriIh0v0VuroWwHrRGS9iBwB3gCutVgzFMtzvS9Rw9OUo7Jvx/4d9Jjag+vevI7a1WrzRb8vGNFxBFUqVrE3tjCSZvQeP7feEEw18bjJmcgbp/FgJdGfBWwOeJzvn1dWa2PMV8aYmcaYxlE+F2NMjjEmzxiTt2PHDgtheUCizo4o7kSJCG+ufJNGoxsxdfVUHm7/MEv7LeWiMy6KLYZy3IX0+oenLLfeEEw18bjgev4iHqqoXzoBNwHjAh73BF4ss85JQHX//12A76w+N9iUNFU3ify+Z6Gyb+uerXL9G9cLw5CWuS1lxbYV9m27HHUwXvg6HK1kqef2MrfV0ScKMdbRtwZmBzx+AHggwnM2ALXK81xJpkTvkrOjpKREXl3+qtR8oqZUGV5Fnlr4lBwtPmrfBsqZsV1yeFQSilerKjdfxGNN9BWB9UB9frmh2rjMOnXg2K9sWwGbAGPlucGmpEn0Io6fHZt2bZKrJ18tDEPaTmgra3eutX8j4X5PH4HbPzxWJct+qMRyTasb+aU65lt8LWiG+Of1B/r7/x8ErPIn8sVAm3DPjTQ5nuiT4FNbXFIsY5aOkRqP1ZBqI6rJi0tejH5YP6vHIVSJ3hhPHrto6TcTVR6ua0ef6MnxgUc8/qldV7BOOkzsIAxDOr7aUdYXro/+RaI5DpMnhy7Vx1jh7oVrbjLea1DxZ/d5Ey7Ra6dmZYXrhcjlvTsVlxQz6otRPDjvQSpWqMg/r/wnfS7sgzEm+heL9jiE2kYMvUp5pO8313aopdzN7vNGOzWLhkfb/H2z8xsum3gZd8++m/aZ7Vk1cBV9L+pbviQP0R+H0u74yoqh/ZlX2tl7vumdckQizxtN9GV57FNbVFLEkwufpPmY5qzZsYZJ101ieo/pZJyUEdsL29EdYoyNyL1yzdX28/ZwqkM4pyT0vAlVp+PkpHX01mzctVFajG0hDEOuf+N6+XHvj/a9uAvGmvNS3bcX7iW4mYc+drZyVaubRE/a6saa/Uf2y6XjL5W3Vr7lG5zb7rgdPg6p+uFPdsFOKy9d1N0qXKLXm7EeJyK+eniv3LmM0pQpvjr5TZt8tUYjRnh6d1JeqNO07L2YUnpD27pwN2M10ScLD7cWUqkj1GmalgbFxb+er6evddrqJhV45c6lSmmhTsfiYr2hHU+a6JOFx1oLqdQU6nQsHTbZxcMoe5om+mThpTZ+qdaOLkAK7zoQ/jS1exhlFSDUXVonJ8db3XiVXa1k4tnaxi1NaRxoUeSWXXeaRxq1eQ7a6kZZFu/WO264aexQCyU37LpKXtrqRlkX72zkdMcwU6bAn/7kSBMPp3ddJbfUaHWT6pWfdgnVLGLjRnuOr5M3jUtL8sGSPMS9hZLeL1dOSY5EX/oB3rjRV2TauNH3WJN99EJlHWOOP749e8LAgdG/vpM3jYP1khYozhnXS/fLVZIJVXnv5BT1zVj9/bR9gt0xDNXXfHkHFnHqbly4kbASdFc0lW9EpvK+JwI2jDDVGVgLrAMGB1meDXztnz4DmgUs2wCsAJaHCyRwijrRh0tEKnplP5GhkqPXLqah9iUtLeWyTqKTrrY4ir+YEj2Qhm8YwHP4ZdzXRmXWaQPU9P9/NbAkYNkGoFak7QROrivRp3pRxEqy98Kx0WwjIs4cBv3SHX+xJvrWwOyAxw8AD4RZvyawJeBx/BN9PM9cTQ7hhwosO9/txyaRF20HCwjhNu1E0tUv3fEXa6K/ERgX8LgnMCrM+n8ts/4PwP+AZUBOmOflAHlAXt26daPfy3h9qLQo4jNgQPg67lQ+NsE4WECItGknkq4TH6N4pAQ3f7mPNdHfFCTRvxhi3Q7AGiA9YN6Z/r+n+at9Lou0TVf9MlaLIr+YPFkkPT1yok/FY1OWgwWESJuOJbTyJrpEX/fisT23f7lPSNUN0NRfl39emNcaBvw10jYdH2Eq8EwOldhStdQaqb4+lY9NIAcLCJE2Xd6EFWuiS2RpOB7XWbd/uY810VcE1gP1A27GNi6zTl1/i5w2ZeZXA2oE/P8Z0DnSNsud6K2eSaHWC3YmV6okUrmyey/jiRap+iaVj00gF5foRcqXdO3epXgm/nCnaXm35fYv93Y0r+wCfOsvsQ/xz+sP9Pf/Pw742d+E8lgzSn9Lna/806rS50aaypXorRY3wq0X6kxOT3dvxVyihSvRp/qxKRWqissldfTlZTnRWcjg8a4GifTFszzbSuoSvRNTuRK91Xch3Hpuv2S7gdsrKks5ddcs2PEpLSy4pNVNeVn6iFk8PxLRIjrY2xDLttx+6qdGoreapMOt5/ZLtlu4uemBiLOfSDecQ3F6fywdVov7n4gyVbgv6eXdlptP/dRI9HaU6N1+yVbWJDrZBn767cwq5Y0ljudwxEQX7hgESORb5IZrbyKkRqK3o46+dLlbL9nKmkRWwVmpI0hkVnE6q4Xafpl+kRJZpkqV8ltqJHqR2FvdKOfZ8d64objoVFZx+j5TuF9Rlzn+KfIj5YRJnUSvvM2uolcii3CRqmsSnVWcLtGLhD8eXuDRq4ImeuUNdiapRH1Y3ZBYA7mhnsJtxyQa5Tl+LrkwaKJX3uB0tUN5uCGxBovJycTjxmNiVbQXKRftqyZ65Q1eLQk6nVjdyKvHJNrChovO2XCJXgcHV+5ROiRk4HB/VatCbi5kZzsXl0odmZm+oTLLCjVwvItGfE+NwcGVd5UO7N6zJ5x4IqSn+z4o9eppkleJFe3Avh4Z8V0TvXJW2YHdCwrg4EH49799JShN8iqRsrN9hYt69awVNjwy4rtW3ShnRftVWSm3mTIFhgyBTZt8JfkRIxwpoISrutFEr5zlojpOpbxM6+iVe3mkjlMpL9NEr5zlkTpOpbxME71yVrQ3v5RSUbOU6I0xnY0xa40x64wxg4MsN8aYkf7lXxtjLrL6XKXIzvbdeC0p0ZY2diltslqhgu/vlClOR6QcFDHRG2PSgJeAq4FGQA9jTKMyq10NNPBPOcDLUTxXKWWnsk1WN270PdZkn7KslOhbAetEZL2IHAHeAK4ts861wCT/L3EXA6cYY86w+FyllJ2GDDn+18XgezxkiDPxKMdZSfRnAZsDHuf751lZx8pzATDG5Bhj8owxeTt27LAQllIqqE2bopuvkp6VRG+CzCvb8DnUOlae65spkisiWSKSVbt2bQthKaWC0iarqgwriT4fODvgcQaw1eI6Vp6rlLKTNllVZVhJ9EuBBsaY+saYysDNwLQy60wDbvW3vrkE2C0iP1p8rlLKTtpkVZVRMdIKIlJkjBkEzAbSgAkissoY09+/fAwwA+gCrAMOAL3CPTcue6KU+kV2tiZ2dYz2daOUUklA+7pRSqkUpoleKaWSnCZ6pZRKcprolVIqybnyZqwxZgcQZNghS2oBO20Mxwt0n5Nfqu0v6D5Hq56IBP21qSsTfSyMMXmh7jwnK93n5Jdq+wu6z3bSqhullEpymuiVUirJJWOiz3U6AAfoPie/VNtf0H22TdLV0SullDpeMpbolVJKBdBEr5RSSc6TiT6Wwcq9ysI+Z/v39WtjzGfGmGZOxGknqwPLG2NaGmOKjTE3JjK+eLCyz8aY9saY5caYVcaYTxMdo90snNsnG2M+NMZ85d/nXk7EaRdjzARjzHZjzMoQy+3PXyLiqQlfd8ffA+cAlYGvgEZl1ukCzMQ3wtUlwBKn407APrcBavr/vzoV9jlgvXn4usq+0em4E/A+nwKsBur6H5/mdNwJ2OcHgSf9/9cGCoHKTscewz5fBlwErAyx3Pb85cUSfSyDlXtVxH0Wkc9E5Gf/w8X4RvPyMqsDy98JTAW2JzK4OLGyz38E3hWRTQAi4vX9trLPAtQwxhigOr5EX5TYMO0jIvPx7UMotucvLyb6WAYr96po96cPvhKBl0XcZ2PMWcD1wJgExhVPVt7n84CaxphPjDHLjDG3Jiy6+LCyz6OAhviGIV0B3CUiJYkJzxG256+II0y5UCyDlXuV5f0xxnTAl+jbxjWi+LOyz88DfxORYl9hz/Os7HNFoAXQETgR+NwYs1hEvo13cHFiZZ+vApYDVwC/AeYYYxaIyJ44x+YU2/OXFxN9LIOVe5Wl/THGNAXGAVeLSEGCYosXK/ucBbzhT/K1gC7GmCIReT8hEdrP6rm9U0T2A/uNMfOBZoBXE72Vfe4FPCG+Cux1xpgfgP8DvkhMiAlne/7yYtVNLIOVe1XEfTbG1AXeBXp6uHQXKOI+i0h9EckUkUzgHWCgh5M8WDu3PwDaGWMqGmOqAhcDaxIcp52s7PMmfN9gMMacDpwPrE9olIlle/7yXIleYhis3Kss7vNDQDow2l/CLRIP9/xncZ+TipV9FpE1xphZwNdACTBORII20/MCi+/zo8BEY8wKfNUafxMRz3ZfbIx5HWgP1DLG5AP/ACpB/PKXdoGglFJJzotVN0oppaKgiV4ppZKcJnqllEpymuiVUirJaaJXSqkkp4leKaWSnCZ6pZRKcv8P/gwvg14rxM4AAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 画出参数更新之前的结果\n",
    "w0 = w[0].data[0]\n",
    "w1 = w[1].data[0]\n",
    "b0 = b.data[0]\n",
    "\n",
    "plot_x = np.arange(0.2, 1, 0.01)\n",
    "plot_y = (-w0 * plot_x - b0) / w1\n",
    "\n",
    "plt.plot(plot_x, plot_y, 'g', label='cutting line')\n",
    "plt.plot(plot_x0, plot_y0, 'ro', label='x_0')\n",
    "plt.plot(plot_x1, plot_y1, 'bo', label='x_1')\n",
    "plt.legend(loc='best')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到分类效果基本是混乱的，我们来计算一下 loss，公式如下\n",
    "\n",
    "$$\n",
    "loss = -(y * log(\\hat{y}) + (1 - y) * log(1 - \\hat{y}))\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 计算loss\n",
    "def binary_loss(y_pred, y):\n",
    "    logits = (y * y_pred.clamp(1e-12).log() + (1 - y) * (1 - y_pred).clamp(1e-12).log()).mean()\n",
    "    return -logits"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "注意到其中使用 `.clamp`，这是[文档](http://pytorch.org/docs/0.3.0/torch.html?highlight=clamp#torch.clamp)的内容，查看一下，并且思考一下这里是否一定要使用这个函数，如果不使用会出现什么样的结果\n",
    "\n",
    "**提示：查看一个 log 函数的图像**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.6314, grad_fn=<NegBackward0>)\n"
     ]
    }
   ],
   "source": [
    "y_pred = logistic_regression(x_data)\n",
    "loss = binary_loss(y_pred, y_data)\n",
    "print(loss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "得到 loss 之后，我们还是使用梯度下降法更新参数，这里可以使用自动求导来直接得到参数的导数，感兴趣的同学可以去手动推导一下导数的公式"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.6307, grad_fn=<NegBackward0>)\n"
     ]
    }
   ],
   "source": [
    "# 自动求导并更新参数\n",
    "loss.backward()\n",
    "w.data = w.data - 0.1 * w.grad.data\n",
    "b.data = b.data - 0.1 * b.grad.data\n",
    "\n",
    "# 算出一次更新之后的loss\n",
    "y_pred = logistic_regression(x_data)\n",
    "loss = binary_loss(y_pred, y_data)\n",
    "print(loss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上面的参数更新方式其实是繁琐的重复操作，如果我们的参数很多，比如有 100 个，那么我们需要写 100 行来更新参数，为了方便，我们可以写成一个函数来更新，其实 PyTorch 已经为我们封装了一个函数来做这件事，这就是 PyTorch 中的优化器 `torch.optim`\n",
    "\n",
    "使用 `torch.optim` 需要另外一个数据类型，就是 `nn.Parameter`，这个本质上和 Variable 是一样的，只不过 `nn.Parameter` 默认是要求梯度的，而 Variable 默认是不求梯度的\n",
    "\n",
    "使用 `torch.optim.SGD` 可以使用梯度下降法来更新参数，PyTorch 中的优化器有更多的优化算法，在本章后面的课程我们会更加详细的介绍\n",
    "\n",
    "将参数 w 和 b 放到 `torch.optim.SGD` 中之后，说明一下学习率的大小，就可以使用 `optimizer.step()` 来更新参数了，比如下面我们将参数传入优化器，学习率设置为 1.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 使用 torch.optim 更新参数\n",
    "from torch import nn\n",
    "w = nn.Parameter(torch.randn(2, 1))\n",
    "b = nn.Parameter(torch.zeros(1))\n",
    "\n",
    "def logistic_regression(x):\n",
    "    return sigmoid(torch.mm(x, w) + b)\n",
    "\n",
    "optimizer = torch.optim.SGD([w, b], lr=1.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "正例数: tensor(91)\n",
      "epoch: 200, Loss: 0.30171, Acc: 0.91000\n",
      "正例数: tensor(90)\n",
      "epoch: 400, Loss: 0.25562, Acc: 0.90000\n",
      "正例数: tensor(89)\n",
      "epoch: 600, Loss: 0.23700, Acc: 0.89000\n",
      "正例数: tensor(89)\n",
      "epoch: 800, Loss: 0.22697, Acc: 0.89000\n",
      "正例数: tensor(89)\n",
      "epoch: 1000, Loss: 0.22077, Acc: 0.89000\n",
      "\n",
      "During Time: 0.338 s\n"
     ]
    }
   ],
   "source": [
    "# 进行 1000 次更新\n",
    "import time\n",
    "\n",
    "start = time.time()\n",
    "for e in range(1000):\n",
    "\n",
    "    # 前向传播\n",
    "    y_pred = logistic_regression(x_data)\n",
    "    loss = binary_loss(y_pred, y_data) # 计算 loss\n",
    "\n",
    "    # 反向传播\n",
    "    optimizer.zero_grad() # 使用优化器将梯度归 0\n",
    "    loss.backward() # 求导\n",
    "    optimizer.step() # 使用优化器来更新参数\n",
    "    \n",
    "    # 计算正确率\n",
    "    mask = y_pred.ge(0.5).float()\n",
    "    acc = (mask == y_data).sum().data / y_data.shape[0]\n",
    "    if (e + 1) % 200 == 0:\n",
    "        print('正例数:',(mask == y_data).sum().data)\n",
    "        print('epoch: {}, Loss: {:.5f}, Acc: {:.5f}'.format(e+1, loss.data, acc))\n",
    "during = time.time() - start\n",
    "print()\n",
    "print('During Time: {:.3f} s'.format(during))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到使用优化器之后更新参数非常简单，只需要在自动求导之前使用 **optimizer.zero_grad()** 来归 0 梯度，然后使用 **optimizer.step()** 来更新参数就可以了，非常简便\n",
    "\n",
    "同时经过了 1000 次更新，loss 也降得比较低了"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面我们画出更新之后的结果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x1ba45379190>"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAAAsTAAALEwEAmpwYAAAvmklEQVR4nO3deXgUVbr48e9JwqqobC4IhPi7OGwCQgRkREFGBQfRUbiiGRU3RMD1qoCMGhcuuA5oUARc0CDKKNcFUVwQFDcIGkA2RVkMIktQZIuQ9Pv7oxNNQnenu9O19vt5nnqS7q6uemt7+9SpU6eMiKCUUsr7UpwOQCmlVGJoQldKKZ/QhK6UUj6hCV0ppXxCE7pSSvlEmlMzbtSokbRo0cKp2SullCctXbp0h4g0DvWZYwm9RYsW5OXlOTV7pZTyJGPMxnCfaZWLUkr5hCZ0pZTyCU3oSinlE5rQlVLKJzShK6WUT1SZ0I0xzxpjthljvgnzuTHGPG6MWWeMWW6M6ZT4MBNoxgxo0QJSUoJ/Z8xwOiKllEqIaErozwN9InzeF2hZOgwBnqp+WBaZMQOGDIGNG0Ek+HfIEE3qSilfqDKhi8jHwM4Io5wPvCBBXwBHGWOOS1SACTVmDOzbV/G9ffuC7yullMclog79eODHcq8LSt87hDFmiDEmzxiTt3379gTMOkabNsX2vlLVoLV7ym6JSOgmxHshn5ohIlNEJFNEMhs3DnnnqrWaN4/t/QTRAzv5aO2eckIiEnoB0Kzc66bATwmYbuKNHQt161Z8r27d4PsW0QM7OWntnnJCIhL6m8Dlpa1dugG7RGRLAqabeFlZMGUKpKeDMcG/U6YE37eIHtj2qXwmNGyYc2dGXqjd0zNHHxKRiAMwE9gCHCRYGr8aGAoMLf3cAJOA74EVQGZV0xQROnfuLMnAGJFg2bziYIzTkflLbq5I3bqh13XZULducDw7pKeHjiE9PfL3cnOD4xgT/GtVvKHWl53rR8UPyJNw+TrcB1YPyZLQ4z2wVWzCrWen1ns8CdPOJKv7pXdFSuj+vVPUJeeTDlTbJ6VoqzLsqvKIp3bPzuo5L1QJqdj5M6G76EqkA9X2SSnahkoWN2iqICsLNmyAQCD4t6ptbmeSdajBl7KYPxO6y65Exnpgq9iFOhOqzO1nRnYmWT1z9Cd/JnQ9n0w6oc6Err/eW2dGdiZZPXP0JxOsY7dfZmamWPYIuhYtgtUslaWnB4vISrnUjBnBE8lNm4Il87FjNcmqiowxS0UkM9Rn/iyh6/mk8iitnlPV4c+ErueTSqkk5M+EDlrUUa7lkha1yofSnA5AqWRS1qK2rBFWWYta0DKHqj7/ltCVciGXtaiNSM8kvEdL6ErZyCstavVMwpu0hK6Ujbxyh6aXziTUnzShK2Ujr7So9cqZhKpIE7pSNrKqRW2i67sbNIjtfeUOWoeulM2yshJbD6313aqMltCV8qDyJfIrrkh8fffOnbG9r9xBE7pKKG3qZr3KvUOXlIQerzr13V65eKsq0oSuEsZF3dD7WqgWKKFUJ/l65eKtqkgTuk85UVLWpm72iKbkXd3k65fukJLtjNGf3ecmucoXySB4gFt9QKakBEvmlRkT7FJHJUa43qFTU4PrWbvdDXLqOLBa8nWfm+ScKilrvas9wlWHTJ+ufdGVl4xnjJrQfcipm0K03tUekapDkq2KobzKyx7qLAZ8fnOUiDgydO7cWVRQbq5IerqIMcG/ubnVm156ukiw8qPikJ5e/VirkuhlUdHLzRWpW7fiNq9bNzm2QahlN8a548BKQJ6Eyaua0B1mxUGYzAd2MnPyh9xp4Za9clK38jiwqzCjCd3FrDoIk6GknAzLGItwJVJjnI7MeuGWvexYsnofsbMQFSmhaysXh2nLkPj4tQVDPMoeLB2uzjgZno3u9HPh7Zx/tVu5GGP6GGPWGmPWGWNGhfj8SGPMW8aYZcaYlcaYK6sbdLLQliHxcWMLBicuSJa/mSuUZLko7fQFedf0Thmu6F42AKnA98AJQE1gGdCm0jh3Ag+W/t8Y2AnUjDRdrXIJ0vru+LitesHO7Vi+qik1NXJVQzLtR05Wwdl5/YLq1KEDpwLzyr0eDYyuNM5o4EnAABnAOiAl0nQ1of9J64Jj57YLgHbFE+qHw00/bMnKLXXo0VS5HA/8WO51Qel75eUArYGfgBXATSJySA2wMWaIMSbPGJO3ffv2KGadHLKygvVselNI9Jw+xa7MrlNuO/pxUbFzS1cJ0SR0E+K9ypfxzgHygSZARyDHGHPEIV8SmSIimSKS2bhx4xhDVepPbjmAyth1LcSOflxUfNxQMIsmoRcAzcq9bkqwJF7elcDs0jOCdcB6oFViQlQqNDccQGXsOmMI9wORmuqOHzblrGgS+hKgpTEmwxhTExgEvFlpnE1AbwBjzDHAX4AfEhmoUm5m1xmD9uOiIqnyEXQiUmyMGQHMI9ji5VkRWWmMGVr6+WTgfuB5Y8wKglU0I0Vkh4VxK+U6iX60XLh5QLAufdMm7VlRVaQ3FimllIdo97lKKZUENKErpZRPaEJXSimf0ISukloyPxBC+U+VrVyU8qvKPTZu3Bh8DdpqRHmTltBV0nJjj41KVYcmdJW0XNPlqVIJogndA3bu38mHP3zodBi+o33RK7/RhO60KK7KPfLZI/ztxb+RNTuLrXu22h6iX7mtx0alqksTupPKP25G5M+rcpWS+l2n38Xdp9/Nq6te5S85f2Fy3mQCh/ZOrGLklh4btaWNSpTkTuhOH0lRXpWrU6MO9/a6l+VDl9O5SWeuf/t6uj/Tnfyf8+2LVVkiyt90paIT7skXVg+OP7HIDc9+i+M5aoFAQHKX5crRDx8tKfemyM3v3Cy/Ff1mX8w+4oZdwG1PXlLuRzWfWORPbmizFsdVOWMMWe2zWDN8Ddd2upaJX06k9aTWzF49u+xxgCpKbtgFtKWNezh9wp4IyZvQ3XAkVeOqXP069ZncbzKfXf0Zjeo24qJZF3HezPNY/8t6i4L1HzfsAtrSxh38UvWVvAndDUdSAq7KdWvajbwheTx69qMs2LCAtk+2Zfyi8RwoOWBh4P7ghl3Ayy1t/FCiLeOGs7WECFcXY/Xg6zr03NxgJagxwb82Vcpu+nWTXPjKhUI20mZSG/l4w8e2zNer3FCHXhaHA7tLtbhl3SVKHJezHEOEOvTkTegi1hxJLtjT31r7lqT/O13IRq58/UrZvnd71N91Y3KxMiY3Lq8X+O1irpeWRxN6ZVYexS7ZM/b8vkdGvj9S0u5LkwYPNpBnvnpGSgIlEb/jgt8iT8TkN/EcDl4q0UbDS/uZJvTyrN5yLtvTV2xdIac9e5qQjfR4tod8s/WbsOO65LfI9TH5SbyHgx+3i1fO1jShl2f1nujCPb0kUCLPfPWMNHiwgaTdlyYj3x8pe37fc8h4Lvstcm1MfhLv7uqlEq3fREroydfKxeq2ai5stpBiUrjq5KtYO2Itl7W/jAc/fZC2T7ZlzrdzKoznhlYf0c5bm/UlRrjdfuPGyK1X3NJtgqokXKa3evBtCV3E9eduCzcslDaT2gjZyD9e/ods+nWTiLiz1OXGmPwk3OGg69u90CqXcjRDiIjI78W/y7hPxkmdB+rIYWMPk0c/e1QOlhx05W+RG2Pyi1CHg8tqDFUlmtArqypDJFEG+WHnD3LujHOFbKTDUx3k8x8/dzqkmCTRprJM+XUYLqHrNQv30IQeiyQswQcCAXl15aty/KPHi8k2ct1b18nOfTudDqtKoTZVzZoiDRtqgo+XC6/pq0oiJfTkuyhaFd/cAxw9YwwXtbmI1cNXc3O3m5n61VRaTWpF7vLc4K++S4XaVAcOQGFhMA15tT8OJ7nwmr6vWN5dQrhMb/Xg2hK6tpOTr376SrpM7SJkI2dOP1PWbF/jdEghRaoi0NJl/LQayxqJOvknQgndSBQlMGNMH2AikApME5HxIcbpCUwAagA7ROSMSNPMzMyUvLy8mH+ALNeiRbBoV1l6OmzYYHc0jikJlDD1q6mM+mAU+4v3M+qvoxjdYzS102o7Hdofwm2qyoyBgD7gSTksUanFGLNURDJDfVZllYsxJhWYBPQF2gCXGGPaVBrnKOBJoL+ItAUGRh+eyyT6nNOjXdKlpqQyNHMoa0asYWCbgdz38X2c9NRJvPf9e06H9odQmyoUbbOu3MCO7pqjqUPvAqwTkR9E5ADwMnB+pXEuBWaLyCYAEdmWuBBtlsg7JnzQyfKxhx9L7oW5fHDZBxgM5+SewyWvXcKW3VucDu2QTdWwIdSoUXEcrf9VbmHLTXLh6mLKBmAAwWqWsteXATmVxplAsBS/AFgKXB5mWkOAPCCvefPmcdVDeYrPmgzsP7hfsj/Kllr315Ijxh0hT3z5hBSXFDsdVgVa/3soXSfuYEcdejQJfWCIhP5EpXFygC+Aw4BGwHfAiZGm69qLoonk0wus3+74Vs564SwhG8mckil5m/OcDiksvyWzWJfHb61wvb49ExF/dRP6qcC8cq9HA6MrjTMKyC73+hlgYKTpJkVC91kJvfzO2Lx5QIaN+1SOfeRYSbk3RW6Ye4P8uv9Xp0OswI/JLNbl8dMu6LftGa/qJvQ04AcgA6gJLAPaVhqnNfBh6bh1gW+AdpGmmxQJ3Ud7YLhFmfLcHhn+9nAx2UaOe+Q4eeWbVyQQCDgdroj4K5mJxLc8kZp2eq2E67ftGa9qJfTg9zkX+Bb4HhhT+t5QYGi5cW4HVpUm85urmmZSJHQR758jlqrqYFpcsFg6Pd1JyEb65PaRdYXrnAxXRPxX4xXP8lTV+ZaXyhdu3p529iZS7YRuxZA0Cd0nojmYikuKZeIXE6Xe/9aT2g/UlgcWPiBFB4sci9lvJbp4lieazre8sj7cuj2rOhFP9Im6JnRVbbEcTAW7CmTgrIFCNtIqp5V8tP4jm6MN8lGNl4jEvzxlpcNwCd0NJdxouHV7VnVsJPqHSBO6qrZ4Dqa5386VjAkZQjZy2ezLZOuerfYFXMonNV5/qM7yuLWEGws3bs+qzl4TXVWkCV0lRDwH074D+2TMh2Okxn01pP74+vJ03tNVPqxaWcOtJVyv0xK6JvSks2rbKjnjuTOEbOTUaafKsp+XOR1SUnJjCdfrtA5dE3p8PH40BgIBef7r56XRQ40k9d5UuW3ebbL7991OhxUXj28KlWBuaeUSVW+LVnBtb4tuVdYvTPkOwOvW9eSTeQv3FTL6w9FM/WoqzY5oxhN9n+D8Vuc7HVbUfLQplAdVq7dF5RI+evBGw7oNmXLeFBZduYijah/FBa9cwPkvn8/GX6PoC9cFfLQpPM2Kjkw92jnqH7SE7hUpKcHqt8o83tn3wZKDTPxyIvcsuAeAe864h1u63UKN1BpVfNM5Pt0UnmLFWZJXzry0hG41O37Wbel70zrhVlGN1Brc1v02Vg9fzd9O+BsjPxhJpymd+HTTp06GG5HHN4UvWHGW5Iszr3CV61YPvrkoaldbMA+3OYsl9NdXvy7NHmsmZCNXv3G17Ni7w/6Aq+DhTeEbVnQD4OauBcpDW7lYyM67NTzatCLWVbT7991y27zbJPXeVGn0UCN57uvnXNPhVxmPbgrfsOKw88qNV5ESutahV5dWqFYp3lW0fOtyrn/7ej778TNOTz+dyX+fTOvGra0LVHmG1qFrHbo1tEK1SvGuovbHtOeTKz9hSr8prNi6gg6TOzDmwzHsO7gv8heV7yXySZFWTtN24YruVg++qXJJtgrVOOoaErGKtu7ZKpf/3+VCNpIxIUPmfjs37kVQysvQOnSLJUuFajUyc6JW0UfrP5JWOa2EbGTArAFSsKsgvgkp5VGRErrWoavotWgBG0Pc/JOeDhs22BbGgZIDPPLZI9z/8f2kpaTxQK8HGN5lOGkpabbFoJRTtA5dJcamTbG9b5GaqTW5s8edrBy2ktOan8bN826my9QuLN682NY4lHIbTegqei67AHxC/ROYe+lc/jPwP2zdu5Vu07ox/O3h/Fr0qyPxKOU0TegqemPHBttxlVe3bvB9hxhjGNBmAKuHr+bGrjcyeelkWuW0YuaKmSS6OtHr/XwoZ9i532hCV9Era9fVsOGf79Wp41w85RxR6wgm9JnA4msW0+zIZlw6+1LOzj2b7wq/S8j0y9oob9wYvBq8cWPwtSZ1FYnd+01yJ3QtcsVn//4//y8sdFVm69ykM19c/QU5fXNYvHkxJz11EvcuuJei4qJqTdcX/Xwo29m+34Rr/mL14HizxWRrPx5JLG0KvXJ/tIj89NtPMujVQUI20vLxlvL+9+/HPS2v9POh3MWK/YYIzRaTt4SuRa6gWM8JLWrpYsXJ0nH1jmPmRTOZ9895BCTAWS+eRdbsLH7e83PM03LZ9WDlEbbvN+EyvdWD4yV0LXIFxVritqCEbsfJ0r4D++Tu+XdLzftrypHjjpQnFz8Z08Oq9YROxcOK/Qa9UzQED1UdWCrWHzYL9lA7N8Wa7WvkzOlnCtlI16ld5estX0f93WS5IVglVqL3G03ooWiRKyiebJrgPdTuk6VAICC5y3Ll6IePlpR7U+SWd2+R34p+s2ZmSiVYpIQeVR26MaaPMWatMWadMWZUhPFOMcaUGGMGJKhGyDq+6FotAeJpW56VFbzVPxAI/q3mOrO7ntEYQ1b7LNYMX8OQTkOY8MUEWk9qzezVs4OlHKU8qsqEboxJBSYBfYE2wCXGmDZhxnsQmJfoIC2T4MTkSS74YXPqfqX6derzVL+n+Ozqz2hUtxEXzbqI82aex/pf1ls7Y6UsEk0JvQuwTkR+EJEDwMvA+SHGuwF4DdiWwPiUHRz+YXP6N6Vb027kDcnjsbMfY8GGBbR9si3jF43nQMkBewJIMnr7h3WiSejHAz+We11Q+t4fjDHHA/8AJkeakDFmiDEmzxiTt3379lhjVT7m9MlSWkoat5x6C6uHr6bPf/Vh9IejOfnpk/lk4yf2BuJzesettaJJ6CbEe5UrGicAI0WkJNKERGSKiGSKSGbjxo2jDFEdQos4lml2ZDNmXzybty55i70H9nL686dz1RtXsWPfDqdD85Rwu6je/mGtaDqQLgCalXvdFPip0jiZwMvGGIBGwLnGmGIReT0RQapyKj/4sKyIA8l5DcAi/U7sR68Wvbj/4/t59PNHeWPtGzx81sMM7jiYFJO89+NFI9Iu6pIemH2rygdcGGPSgG+B3sBmYAlwqYisDDP+88AcEXk10nT1ARdxcslDJpLJym0rGfr2UBZtWsRpzU/jqb8/Rbuj2zkdlmtF2kVBd9/qqtYDLkSkGBhBsPXKamCWiKw0xgw1xgxNbKiqSlrEsV3bo9uycPBCnun/DKu2r+Lkp09m1Aej2Htgr9OhuVKkXdSFPTD7ij6Czmu0hO6oHft2cMf7d/Bc/nOkH5lOzrk59Duxn9NhuUpVu+iMGcE6802bgvcajB2rtYWx0EfQ+YkWcRzVqG4jnj3/WRYOXshhNQ/jvJnnceErF/Ljrh+r/nKSqGoXdbpFk59pQvcapxttKwBOTz+dr6/7mnG9x/HuundpPak1j33+GMWBYqdDc5zuos7RKhelqmn9L+sZ8c4I5n43lw7HdGByv8l0a9rN6bCUT2mVi1IWyqifwZxL5vDaf7/Gjn076P5Md4bOGcov+39xOjSVZDShK5UAxhgubH0hq4ev5uZuN/P0c7tp1GQPJkVITxe990vZIpobi5RSUapXqx6ddzxG7XdKKNqfCgRbc1xzbQBI0XpkZSktoSvruKGLAgdiGDOGP5J5maL9KQy/9Vf2H9wf5ltKVZ8m9GRldaJzQy9MDsUQ7saaXduO4KSnTuK979+zdP4qeWlCT0Z2JDo39MLkUAzhHsxxdJPfSTEpnJN7DoNeHcSW3VssjUMlH03oyciOROeGLgociiHcjTWPPVSH5dcvJ/uMbF5f8zqtJrUiZ3EOJYGInZQqFTVN6MnIjkRn93PlXBRDpBtraqfV5p6e97Di+hV0Pb4rN7xzA92e6cbSn5ZaGpNKDprQk5Edic7pLgpmzIA9ew5936YYqrq9vWXDlsz75zxmXjSTgt8K6DKtCze+cyO7inZZHpvyL28ldDe0mvCDSMk2UevYyfu/y64RFBZWfL9hQ1fdg26MYVC7QawevprrM68nZ3EOrSe1ZtbKWfqwahUfEXFk6Ny5s8QkN1ekbl2R4GW84FC3bvB9FbvcXJH0dBFjgn9zc/2zjtPTKy5D2ZCe7nRkES0uWCydnu4kZCN9cvvIusJ1ToekXAjIkzB51Tt9uWi3sdYLt44bNoQdHnoEW0pKMIVXZkywDsTFSgIlPLnkScbMH8PBwEH+1eNf3Nb9Nmql1XI6NOUS/ujLxQ2tJvwu3LosLPRW9ZYbLsjGKTUllRu63sCaEWvo/5f+/Oujf9Hx6Y4s2LDA6dCUB3gnoXv4IPWMSOvSS0/xdfqCbAI0qdeEVwa8wjtZ73Cg5AC9pvfiitevYNvebU6HViW91OWgcHUxVg9ah+5Cubmh654hWNfuJaGuEXjUvgP7ZMyHY6TGfTWk/vj6MiVvipQESpwOKyQ9TK1HhDp07yR0EV8dpK7VsKEnLygmg1XbVskZz50hZCOnTjtVlv28rMrv2H3IePR6tKdESujeqXIBa59dpeeJQRMner66wq9aN27NR1d8xPQLpvPdzu/o9HQnbn/vdvYcCNHeHme6stFLXQ4Ll+mtHuIqoVtFzxMrqlysu/56PTOKxIEzx8J9hXLtm9cKF14iqfV/FGMCh8zaidKyltCth2+qXKyie2F4+mMXmYPrJzdXpFad4gqzrlOn5I9ZGxN6t7bycojuLtaLlNC90w7dSh5ut2ypGTPgiiugJETnUdr+P8jB+yPCzfqoY3axbXNdWv6/Go6ENmNGsFHUpk3BhlNjx7rm5lxf8Ec7dCtpk8hDlVXAhkrmoJWiZRysNA43i1+31qPTlE5ccetaRy6HWHmpqzKrLn159ZKaJnTwRbvlhAvVxW55yfxjV56DhYFws2jcpIhdRbu475dWdBk6labNSmzvTscOVl30dcOzWeIWri7G6sFVdegi2iSysnAVsFopWpHDdejhZr37991y+3u3S+q9qdLooUby3NfPSSAQiGnabj8crLr05fZLalT3oijQB1gLrANGhfg8C1heOnwGdKhqmo4ndC/ssU4Kt1enpuq6qszBfamqWS//ebl0f6a7kI2c/tzpsnLbyqim6YULm1Zd9HXiYnIsqpXQgVTge+AEoCawDGhTaZzuQP3S//sCX1Y13bgTeiIOHq/ssU7SdeQbJYESmbp0qtQfX1/S7kuT0R+Mlr0H9oYd3+0l1DJaQo8voZ8KzCv3ejQwOsL49YHNVU037jtFE5Fk3L7F3ELPYnxl255tcsX/XSFkIy0mtJC3v3075HiJLqFatRuFSgdlsVdnPm4vy1Q3oQ8AppV7fRmQE2H828qPX+mzIUAekNe8efPYlyTWRBxuT3L7OZVSFvpo/UfSKqeVkI0MmDVACnYVVPg8keUdq5Nj2SFePpknYj5uLstUN6EPDJHQnwgzbi9gNdCwqunGVUKPJRFH2pO0hK6qy81HfBR+L/5dHlj4gNR+oLYc/r+Hy78//7ccLDkoIjEk4SjWgV2HWjId0rZUuQDtS+vaT6xqmhJvQo9lq0Ua1+3nVCp6TiRWH+0/3+/8Xvrk9hGykZMnnyxfFnwpIlGs1ijXgV0nw8l00l3dhJ4G/ABklLso2rbSOM1LW8B0r2p6ZYPldehVbWGPl7CUOJdYfVYcDAQCMuubWdLk0SZiso0MmzNMftn/S+QvRbkOtISeeIlotngu8G1pCXxM6XtDgaGl/08DfgHyS4ewMywbLG/lkkxbOFk5tY3dUhxMcKFkV9EuuemdmyTl3hQ55uFj5KXlL4Vvux7lOrDrN9dHJ01VqnZCt2KwvB16Mm3hZGV3Yo10/cXuwoKF+/fSn5ZK5pRMIRv52wt/k293fHvoSDH8mNp1MpwsJ93JmdBFkmcLJys7S+ihEqiThQWLl724pFgmLZ4kR4w7QmrdX0uyP8qWooNFf46gBSbHJG9CV/5mZ1KpqmRudyKz6ezkp99+kkGvDhKykZaPt5T3v3//zw9zcys+4aphQ03oNoiU0LVzLuVdWVnB3qbS07G896lwXRsaY32XgqHY1CnYcfWOY+ZFM3nvn+8hCGe9eBZZs7P4ec/PwRH27/9z5MJCD/Vi5U/aH7pS0XCw3/OQyroELN8jZt26lnanWFRcxPhF4xm3aBx10urw48RU6v2889ARta98S2l/6Mp9vNbhtNu6WLbz7KRU7bTaZPfMZsX1K+jcpDOHhUrmoH3lOylcXYzVg9ahJzGvXlDTi+x/CAQCsvu4hhL2moLbeXhb4pVH0B08eJCCggKKioociSnZ1K5dm6ZNm1KjRg17Z+y26gsVnxkzkGuvxZSrRy+uXZPUac9i3PwUDQeqqxIpUpWLqxL6+vXrqVevHg0bNsQY40hcyUJEKCwsZPfu3WRkZNg7c32Gq3+UPkBUNm1iS/00but5kF0XnUtO3xwy6tu8X0UrngKFix6U6pk69KKiIk3mNjHG0LBhQ2fOhvQZrv5R+gBREwhw9PZ9ZN72KAs3LKTtk20Z98k4DpQccDrCQ8X6HFgPPZPOVQkd0GRuI8fWtdsuMKqESEtJ49ZTb2X18NX0bdmXO+ffyclPn8zHGz92OrSKYi1QhHq+7r59wfddxnUJXSUBB1poKPs0O7IZr/33a7x1yVvsPbCXM54/g6veuIod+3Y4HVpQrAWKWEv0DtKEXg0bNmzgpZde+uN1fn4+c+fO/eP1m2++yfjx4xMyr8GDB/Pqq68CcM0117Bq1aqETNcxpafqBALO3JijLNfvxH6sHLaSUX8dxYvLX+QvOX/h2a+fJSAOXyeJtUDhoSpCTejVUFVC79+/P6NGjUr4fKdNm0abNm0SPl2lEu2wmocx7m/jyL8unzaN23D1m1dzxvNn8M22b5wNLJYChYeqCNOcDiCcm9+9mfyf8xM6zY7HdmRCnwkRx3nhhRd45JFHMMbQvn17XnzxRQYPHky/fv0YMGAAAIcffjh79uxh1KhRrF69mo4dO3LJJZcwadIk9u/fz6JFixg9ejT79+8nLy+PnJwcBg8ezBFHHEFeXh4///wzDz30EAMGDCAQCDBixAgWLlxIRkYGgUCAq6666o95hdKzZ08eeeQRMjMzOfzww7npppuYM2cOderU4Y033uCYY45h+/btDB06lE2lp4UTJkzgr3/9a8LWpVKxaHt0WxYOXsj0/Onc/v7tnPz0yfzPqf/DXaffxWE1D3M6vMjKkr1LWrlEoiX0clauXMnYsWOZP38+y5YtY+LEiRHHHz9+PD169CA/P5+RI0dy3333cfHFF5Ofn8/FF198yPhbtmxh0aJFzJkz54+S++zZs9mwYQMrVqxg2rRpfP755zHFvHfvXrp168ayZcs4/fTTmTp1KgA33XQTt9xyC0uWLOG1117jmmuuiWm6Cee1O0NVwqWYFK48+UrWjFjD5e0v58FPH6Ttk215a+1bTodWNY9UEbq2hF5VSdoK8+fPZ8CAATRq1AiABg0aJHT6F1xwASkpKbRp04atW7cCsGjRIgYOHEhKSgrHHnssvXr1immaNWvWpF+/fgB07tyZ999/H4APPvigQj37b7/9xu7du6lXr16CliYGlW/kKGv2Ba49MJR1GtVtxDPnP8PgjoMZ+vZQ+r/cn3+0+gcT+0yk2ZHNnA7P07SEXo6IhGzKl5aWRqD0hhcR4cCB+NrW1qpVq8K8yv+NV40aNf6IOTU1leLiYgACgQCff/45+fn55Ofns3nzZmeSOXiq2ZeyT4/0Hnx93deM6z2Od9e9S+tJrXns88coDhQ7HZpnaUIvp3fv3syaNYvCwkIAdu4Mdj7UokULli5dCsAbb7zBwYMHAahXrx67d+/+4/uVX0fjtNNO47XXXiMQCLB161YWLFiQgCWBs88+m5ycnD9e5+fnJ2S6cfFQsy9lr5qpNRl12ihWDltJzxY9+Z/3/ofMKZl8UfCF06F5kib0ctq2bcuYMWM444wz6NChA7feeisA1157LQsXLqRLly58+eWXHHZY8CJO+/btSUtLo0OHDvz73/+mV69erFq1io4dO/LKK69ENc+LLrqIpk2b0q5dO6677jq6du3KkUceWe1lefzxx8nLy6N9+/a0adOGyZMnV3uacfNQsy/ljIz6Gbx1yVu89t+vsWPfDro/052hc4byy/5fnA7NW8L12mX1EKq3xVWrViWmOzKP2b17t4iI7NixQ0444QTZsmWLbfO2ZZ17tXdF5Yjfin6TW969RVLuTZGjHz5aXlz2YviHVSch9IlF7tavXz86duxIjx49uOuuuzj22GOdDimx9M5QFYN6terx2DmPkXdtHi2OasFl/3cZvV/ozdoda50OzfVc1dvi6tWrad26tSPxJCtd58rNSgIlTFk6hdEfjmZ/8X5G/XUUo3uMpnZabadDc4xneltUSqnyUlNSuf6U61k7Yi0D2wzkvo/vo92T7Xjv+/ecDs2VNKErpVzvmMOPIffCXD647ANSU1I5J/ccBr06iC27tzgdmqtoQldKeUbvE3qzfOhy7ut5H6+veZ1Wk1oxafEkSgIlTofmCprQlVKeUiutFnedcRffDPuGrsd3ZcQ7I+j2TDeW/rTU6dAcF1VCN8b0McasNcasM8Yc0n2gCXq89PPlxphOiQ81BO0fRCW7JD4G/qvBfzHvn/N4+aKXKfitgC7TunDjOzeyq2iX06E5psqEboxJBSYBfYE2wCXGmMp9t/YFWpYOQ4CnEhznoRx+LNT06dNp2bIlLVu2ZPr06bbMU6kKPPRoNKsYY7i43cWsGb6GYZnDyFmcQ+tJrZm1cla1u9XwpHAN1MsG4FRgXrnXo4HRlcZ5Grik3Ou1wHGRplvtG4vS0yveqFI2pKdHP404FRYWSkZGhhQWFsrOnTslIyNDdu7cafl8rZCsN3P5goPHgFst2bxEOj3dSchG+uT2kXWF65wOKeGo5o1FxwM/lntdUPperONgjBlijMkzxuRt3749illHYEH/IEuWLKF9+/YUFRWxd+9e2rZtyzffHNoR/7x58zjrrLNo0KAB9evX56yzzuLdd9+Ne75KxUX7yDlEZpNMFl+zmMf7PM6nmz6l3VPteODjB/i9+HenQ7NFNAk91JOEK5/LRDMOIjJFRDJFJLNx48bRxBeeBf2DnHLKKfTv359//etf3HHHHfzzn/+kXbt2h4y3efNmmjX7s5vPpk2bsnnz5rjnq1RctI+ckFJTUrmh6w2sGbGG8048j7s+uouOT3dkwYYFTodmuWgSegFQvpPipsBPcYyTWBY9Furuu+/m/fffJy8vjzvuuCPkOBKibi5Ut7tKWcpDj0ZzQpN6TZg1cBZzL53L78W/02t6L654/Qq27d3mdGiWiSahLwFaGmMyjDE1gUHAm5XGeRO4vLS1Szdgl4hY2+Lfov5Bdu7cyZ49e9i9ezdFRUUhx2natCk//vhnDVNBQQFNmjSp1nyVipn2kROVvi37snLYSsb0GMPMFTNpldOKqUunOv+wagtE1ZeLMeZcYAKQCjwrImONMUMBRGSyCRZPc4A+wD7gShHJCzc9cG9fLv3792fQoEGsX7+eLVu2VOhTvMzOnTvp3LkzX331FQCdOnVi6dKlCX/CkR3csM6Vssvq7asZNncYCzYs4NSmpzK532TaH9Pe6bBiEqkvl6geQScic4G5ld6bXO5/AYZXJ0g3eOGFF0hLS+PSSy+lpKSE7t27M3/+fM4888wK4zVo0IC77rqLU045BQhW03gxmSuVbFo3bs38y+fz4vIXue292+j0dCdu6XYL9/S8h8NrHu50eNWmvS0mOV3nKlnt3L+TUR+MYupXU2l2RDOe6PsE57c63+mwqqS9LSqlVCUN6jRgynlT+PSqTzmq9lFc8MoF9J/Zn42/bnQ6tLhpQo9gxYoVdOzYscLQtWtXp8NSSiVQ92bdWTpkKQ+f9TAfrv+QNk+24aFPH+JgyUGnQ4uZVrkkOV3nSv1p065N3PjOjbyx9g3aHd2Op/7+FKc1P83psCrQKhellIpC8yOb8/qg13n94tfZVbSLHs/14Jo3r6FwX6HToUVFE7pSSlVyfqvzWTV8FbedehvP5z9Pq0mteD7/edd3+KUJXSmlQji85uE8fPbDfHXdV5zY8ESufONKek7vyartq5wOLSxPJ/Qk7gpaKWWT9se055MrP2FKvyms2LqCDpM7cOeHd7Lv4D6nQzuEZxO6011B9+nTh6OOOop+/frZM0OllGNSTArXdr6WtSPWknVSFuMWjaPdk+2Y+93cqr9sI88m9DFjYF+lH8h9+4Lv2+H222/nxRdftGdmSilXaHxYY56/4Hk+uuIjaqXV4u8v/Z2B/xnI5t/c0duqZxO6FV1BR9sfOkDv3r2pV69e/DNTSnlWzxY9WTZ0GWPPHMucb+fQalIrJn4xkeJAsaNxeTahW9EVdLT9oSulVM3UmtzZ405WDlvJac1P4+Z5N9NlahcWb17sWEyeTehWdQUdTX/oSilV5oT6JzD30rn8Z+B/2Lp3K92mdWP428P5tehX22PxbEK3qivoaPpDV0qp8owxDGgzgNXDV3Nj1xuZvHQyrXJa8dKKl2xtu+7ZhA7B5L1hAwQCwb+J6Nd/yJAh3H///WRlZTFy5MjqT1AplTSOqHUEE/pMYMm1S2h2ZDOyZmdxdu7ZfFf4nS3z93RCT7Ty/aGPGjWKJUuWMH/+/JDj9ujRg4EDB/Lhhx/StGlT5s2bZ3O0Sim36nRcJ764+gsmnTuJxZsXc9JTJzE5b3LVX6wm7Zwryek6V8paW3Zv4db3buWak6+h9wm9qz29aj+xSCmlVHyOq3ccMy+aacu8NKFHsGLFCi677LIK79WqVYsvv/zSoYiUUio81yV0ESH4zGnnnXTSSeTn5zsdhmXc3nOcUio2rrooWrt2bQoLCzXR2EBEKCwspHbt2k6HopRKEFeV0Js2bUpBQQHbt293OpSkULt2bZo2bep0GEqpBHFVQq9RowYZGRlOh6GUUp7kqioXpZRS8dOErpRSPqEJXSmlfMKxO0WNMduBjXF+vRGwI4HheIEuc3LQZU4O1VnmdBFpHOoDxxJ6dRhj8sLd+upXuszJQZc5OVi1zFrlopRSPqEJXSmlfMKrCX2K0wE4QJc5OegyJwdLltmTdehKKaUO5dUSulJKqUo0oSullE+4OqEbY/oYY9YaY9YZY0aF+NwYYx4v/Xy5MaaTE3EmUhTLnFW6rMuNMZ8ZYzo4EWciVbXM5cY7xRhTYowZYGd8VohmmY0xPY0x+caYlcaYhXbHmGhR7NtHGmPeMsYsK13mK52IM1GMMc8aY7YZY74J83ni85eIuHIAUoHvgROAmsAyoE2lcc4F3gEM0A340um4bVjm7kD90v/7JsMylxtvPjAXGOB03DZs56OAVUDz0tdHOx23Dct8J/Bg6f+NgZ1ATadjr8Yynw50Ar4J83nC85ebS+hdgHUi8oOIHABeBs6vNM75wAsS9AVwlDHmOLsDTaAql1lEPhORX0pffgF4vf/baLYzwA3Aa8A2O4OzSDTLfCkwW0Q2AYiI15c7mmUWoJ4JPuHmcIIJvdjeMBNHRD4muAzhJDx/uTmhHw/8WO51Qel7sY7jJbEuz9UEf+G9rMplNsYcD/wDsP6x6faIZjufCNQ3xiwwxiw1xlxuW3TWiGaZc4DWwE/ACuAmEQnYE54jEp6/XNUfeiWhnkNXuY1lNON4SdTLY4zpRTChn2ZpRNaLZpknACNFpMQtjyespmiWOQ3oDPQG6gCfG2O+EJFvrQ7OItEs8zlAPnAm8P+A940xn4jIbxbH5pSE5y83J/QCoFm5100J/nLHOo6XRLU8xpj2wDSgr4gU2hSbVaJZ5kzg5dJk3gg41xhTLCKv2xJh4kW7b+8Qkb3AXmPMx0AHwKsJPZplvhIYL8EK5nXGmPVAK2CxPSHaLuH5y81VLkuAlsaYDGNMTWAQ8Galcd4ELi+9WtwN2CUiW+wONIGqXGZjTHNgNnCZh0tr5VW5zCKSISItRKQF8CowzMPJHKLbt98Aehhj0owxdYGuwGqb40ykaJZ5E8EzEowxxwB/AX6wNUp7JTx/ubaELiLFxpgRwDyCV8ifFZGVxpihpZ9PJtji4VxgHbCP4C+8Z0W5zHcDDYEnS0usxeLhnuqiXGZfiWaZRWS1MeZdYDkQAKaJSMjmb14Q5Xa+H3jeGLOCYHXESBHxbLe6xpiZQE+gkTGmALgHqAHW5S+99V8ppXzCzVUuSimlYqAJXSmlfEITulJK+YQmdKWU8glN6Eop5ROa0JVSyic0oSullE/8f132WBrC4DU/AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 画出更新之后的结果\n",
    "w0 = w[0].data[0]\n",
    "w1 = w[1].data[0]\n",
    "b0 = b.data[0]\n",
    "\n",
    "plot_x = np.arange(0.2, 1, 0.01)\n",
    "plot_y = (-w0 * plot_x - b0) / w1\n",
    "\n",
    "plt.plot(plot_x, plot_y, 'g', label='cutting line')\n",
    "plt.plot(plot_x0, plot_y0, 'ro', label='x_0')\n",
    "plt.plot(plot_x1, plot_y1, 'bo', label='x_1')\n",
    "plt.legend(loc='best')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到更新之后模型已经能够基本将这两类点分开了"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "前面我们使用了自己写的 loss，其实 PyTorch 已经为我们写好了一些常见的 loss，比如线性回归里面的 loss 是 `nn.MSE()`，而 Logistic 回归的二分类 loss 在 PyTorch 中是 `nn.BCEWithLogitsLoss()`，关于更多的 loss，可以查看[文档](http://pytorch.org/docs/0.3.0/nn.html#loss-functions)\n",
    "\n",
    "PyTorch 为我们实现的 loss 函数有两个好处，第一是方便我们使用，不需要重复造轮子，第二就是其实现是在底层 C++ 语言上的，所以速度上和稳定性上都要比我们自己实现的要好\n",
    "\n",
    "另外，PyTorch 出于稳定性考虑，将模型的 Sigmoid 操作和最后的 loss 都合在了 `nn.BCEWithLogitsLoss()`，所以我们使用 PyTorch 自带的 loss 就不需要再加上 Sigmoid 操作了"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 使用自带的loss\n",
    "criterion = nn.BCEWithLogitsLoss() # 将 sigmoid 和 loss 写在一层，有更快的速度、更好的稳定性\n",
    "\n",
    "w = nn.Parameter(torch.randn(2, 1))\n",
    "b = nn.Parameter(torch.zeros(1))\n",
    "\n",
    "def logistic_reg(x):\n",
    "    return torch.mm(x, w) + b\n",
    "\n",
    "optimizer = torch.optim.SGD([w, b], 1.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.6145)\n"
     ]
    }
   ],
   "source": [
    "y_pred = logistic_reg(x_data)\n",
    "loss = criterion(y_pred, y_data)\n",
    "print(loss.data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch: 200, Loss: 0.30061, Acc: 0.87000\n",
      "epoch: 400, Loss: 0.25528, Acc: 0.89000\n",
      "epoch: 600, Loss: 0.23683, Acc: 0.89000\n",
      "epoch: 800, Loss: 0.22686, Acc: 0.89000\n",
      "epoch: 1000, Loss: 0.22071, Acc: 0.89000\n",
      "\n",
      "During Time: 0.221 s\n"
     ]
    }
   ],
   "source": [
    "# 同样进行 1000 次更新\n",
    "\n",
    "start = time.time()\n",
    "for e in range(1000):\n",
    "    # 前向传播\n",
    "    y_pred = logistic_reg(x_data)\n",
    "    loss = criterion(y_pred, y_data)\n",
    "\n",
    "    # 反向传播\n",
    "    optimizer.zero_grad()\n",
    "    loss.backward()\n",
    "    optimizer.step()\n",
    "\n",
    "    # 计算正确率\n",
    "    mask = y_pred.ge(0.5).float()\n",
    "    acc = (mask == y_data).sum().data / y_data.shape[0]\n",
    "    if (e + 1) % 200 == 0:\n",
    "        print('epoch: {}, Loss: {:.5f}, Acc: {:.5f}'.format(e+1, loss.data, acc))\n",
    "\n",
    "during = time.time() - start\n",
    "print()\n",
    "print('During Time: {:.3f} s'.format(during))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到，使用了 PyTorch 自带的 loss 之后，速度有了一定的上升，虽然看上去速度的提升并不多，但是这只是一个小网络，对于大网络，使用自带的 loss 不管对于稳定性还是速度而言，都有质的飞跃，同时也避免了重复造轮子的困扰"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下一节课我们会介绍 PyTorch 中构建模型的模块 `Sequential` 和 `Module`，使用这个可以帮助我们更方便地构建模型"
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "07b2cea26089ea0ebec7cc3a83022d31e9f80d0db55f432f89380becb3d80933"
  },
  "kernelspec": {
   "display_name": "mx",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
